Andrew Ng discusses the foray into Machine Learning and Deep Learning, starting the MOOC (Massive Open Online Course) revolution, Google Brain, launching AI Fund, Landing AI, and Deeplearning.ai, AI threats, and more with Lex Fridman.
Andrew Ng (@andrewyng) is a British-born American businessman, computer scientist, and investor. He is the co-founder of Coursera, an online learning platform that offers MOOCs, specializations, and degrees. He is an adjunct professor of the Computer Science Department at Stanford University where his machine learning course CS229 is one of the most popular courses offered on campus. Andrew Ng has co-founded and led Google Brain and has served as Vice President and Chief Scientist at Baidu. He is also the founder of Landing AI, co-founder of Deeplearning.ai, and has launched an investment fund for backing artificial intelligence startups called AI Fund.
Enjoy the takeaways from the conversation drafted concisely in this Artificial Intelligence Podcast Andrew Ng Show Notes.
- Andrew Ng started coding at the ages of 5 and 6 when he copied codes into his computer to create and play his own shoot-’em-up games.
- He first thought about an automation system during his internship in Singapore where his job involved taking a lot of photocopies. He wanted to free up the time used for photocopying to do other productive things.
- While teaching Machine Learning (ML) classes at Stanford University, he recorded his sessions and felt that his recordings of the previous year could be watched by students instead of him having to say the same jokes and repeat the same sessions again, every year.
- Andrew Ng wanted to help anyone with interest in Machine Learning and hence with a group of students started recording videos and publishing them on the internet.
- A lot of the videos of Machine Learning that were shot in the initial days were between 10 pm and 3 am when Andrew, rather than returning home after dinner, went straight to his office to record classes.
- There were earlier prototypes of websites built from which the team learned a lot which eventually led to the birth of Coursera.
- According to Andrew Ng, Data Science and Machine Learning may be an easier entrée into the developer world than Software Engineering.
- Andrew Ng along with Pieter Abbeel, Adam Coates, and Morgan Quigley used reinforcement learning in an autonomous helicopter at Stanford that showcased extreme aerobatics like flying upside down, flying in loops, and other maneuvers. This was in 2008.
- As much as theoretical learning interests Andrew Ng, he likes to see the practical application of principles to things to make them work.
- One of the earlier breakthroughs in Deep Learning was coming to understand the importance of scale – the bigger the data set used for training, the better the result.
- While leading Google Brain, Andrew Ng helped the Google Speech Team with Deep Learning to achieve more accurate speech recognition. He also helped the Google Maps team to accurately locate houses within Google Maps.
- Deeplearning.ai was founded to help people with courses that help them break into AI. Basic programming knowledge and high school mathematics are sufficient to take up Deep Learning. However, Andrew Ng suggests a Machine Learning to Deep Learning transition will be easier for students.
- If the research budget is not a constrain and the focus is only on the long-term impact, Andrew Ng would involve himself completely into Unsupervised Learning.
- Andrew Ng wants developers taking up jobs to know beforehand the team they will be working with because a person will learn faster if he/she is working with a team of great people.
- AI Fund helps in building successful Artificial Intelligence Startups from scratch.
- Landing AI helps already established companies improve with help from Artificial Intelligence.
- Andrew Ng believes that no matter how long it takes, we will get to an Artificial General Intelligence.
- He wants people to not worry about the AI existential threats to humanity because that is a problem far, far into the future, and insists we focus on solving the small problems that exist today.
- Two things Andrew Ng is proud of is his daughter and helping other people achieve their dreams.
Andrew Ng Quotes
- “If you want to make a breakthrough you sometimes have to have conviction and do something before it’s popular since that lets you have a bigger impact.”
- “Similar to learning mathematics, I think one of the challenges of deep learning is that there are a lot of concepts that build on top of each other.”
- “If you are working with great people, you’ll learn faster.”
- “In Silicon Valley, a lot of startup failures come from building out products that no one wanted.”
- “I think the meaning of life is helping others achieve whatever are their dreams and then also to try to move the world forward by making humanity more powerful as a whole.”
Andrew Ng’s Inspiration
Andrew Ng started coding when he was 5 to 6 years old when he would copy code into his computer to create his own shoot-’em-up games.
He really considered the idea of automation on his internship in Singapore where his job involved taking a lot of photocopies from the photocopying machine. He thought how a software program can automate the job that will free up his time to do other productive work.
Teaching at Stanford and MOOCs
Andrew Ng started teaching Machine Learning at Stanford University where he recorded his classes. He realized that he was repeating every year the same jokes and information which were already recorded and could be used by students. He wanted to bring Machine Learning education to the masses and along with a group of students, started recording videos and published them on the internet.
- Andrew Ng would not return home after his dinner, but would rather go back to his office to record classes until early morning.
- A lot of his videos were shot between 10 pm and 3 am.
- Andrew Ng and his team of students initially uploaded the videos on YouTube.
- The team had 4-5 website versions built for testing and were learning a lot from it. This eventually led to the creation of Coursera.
- Andrew Ng prefers using a Whiteboard and Marker for his classes and Lex Fridman likes his approach of not covering a lot of content but clearly explaining a few simple ideas in each class.
Reinforcement Learning and Deep Learning
- Andrew Ng, along with Pieter Abbeel, Adam Coates, and Morgan Quigley at Stanford University used reinforcement learning in an autonomous helicopter flight to make an RC helicopter perform extreme aerobatics under computer control.
- As much appreciation as Andrew Ng has for theoretical learning, he likes to see the practical application of his work to make things work.
- According to Andrew, the early importance to Unsupervised Learning proved to be a wrong path to take and the Importance of Scale proved to be the right one. Importance of Scale means bigger the data sets we use to train a learning algorithm, the better are its results.
- Andrew says both better architectures for learning and bigger and better data sets are important. However, the preference of one over the other should depend on the problem at hand. If we are already surpassing the human level of performance, better architectures can help but if we are below the human level of performance, then bigger and better data sets can help.
Andrew Ng co-founded Deeplearning.ai with Kian Katanforoosh to create courses that help people break into Artificial Intelligence.
- The pre-requisites for Deep Learning, according to Andrew are understanding of basic programming and knowledge of high school mathematics.
- Andrew Ng feels that the transition from Machine Learning to Deep Learning will be easier for students.
- Andrew says that Deep Learning is similar to mathematics in the sense that a lot of concepts build on top of each other and one has to understand the fundamentals before moving to advanced concepts.
- The official time to complete the courses is 16 weeks but it can be done at one’s own pace. However, certain students complete the courses well before the official time.
- Andrew Ng advises students to first involve themselves with courses during the first two years of learning Deep Learning and then move on to projects, blog posts, research papers, and more.
- He advises students to take the first crucial step and then take small steps in their Deep Learning journey.
- If after a certain level of proficiency in Deep Learning, if one needs to take the academic route and get a Ph.D., then Andrew says that Top Schools in the United States provide a very good experience. However, if one wants to take up a job, Andrew wants them to ask their employer about the team they will be working with inside the company because one can learn faster working with great people.
Andrew Ng launched the AI Fund to back the creation of successful Artificial Intelligence Startups from scratch.
- According to Andrew, a lot of startups in Silicon Valley fail because they build products that no one wants.
- The ultimate goal of the AI fund is to find an automated path for the creation of successful Artificial Intelligence startups from scratch. They are in the early stages of this goal.
Experiences at Google Brain and Baidu
Andrew Ng co-founded and led the Google Brain team inside Google. He also served as Vice President and Chief Scientist at Baidu.
- While leading Google Brain, Andrew helped Google Speech Team to improve the accuracy of the speech recognition algorithm. He also helped Google Maps Team to accurately locate houses within Google Maps.
- At Baidu, one part of Andrew’s responsibility was to find new business opportunities using the AI capabilities of the company. The self-driving car team and smart speaker team came out of Andrew’s group at Baidu. They introduced smart speakers before Amazon came up with Alexa smart speakers.
Andrew Ng feels that though Software and Internet Companies have made use of AI to a certain extent, industries like Manufacturing, Agriculture, Healthcare, Logistics, Transportation, etc. have very few people working on bringing AI into industrial processes.
- Andrew Ng recommends starting out small when it comes to implementing AI in a new industry.
- He says that results from test sets can be completely different from the deployment system.
- He wants companies to think about the changes that will be needed when bringing in AI into the process. Sometimes they might need to add more processes around the AI process to make it foolproof.
Artificial General Intelligence (AGI)
- Andrew Ng believes that we will get to Artificial General Intelligence but doesn’t know when we will get there.
- He says that instead of worrying about the existential threats of AI which is at a very early stage now, it is important for us to first solve the small problems of today with it.
Regrets and Being Proud
- Andrew Ng says that he has made a lot of mistakes which in hindsight were easy to resolve. But in reality, the process of discovery takes longer while working on something and things don’t seem as easy as they do in hindsight.
- He is most proud of his daughter, Nova, and the fact that he can help other people achieve their dreams, thereby moving the world forward.
- Even Biologists, Physicists, and Engineers sitting on large data sets are realizing the need for Machine Learning to make sense of the data.
- According to Andrew Ng, every programmer is self-taught.
- Andrew Ng would pursue Unsupervised Learning if the research budget was unlimited and the focus is only on the long-term impact.
- Getting into the habit of learning and putting a sustained effort for a long time will take one far in his/her learning journey.
- Andrew Ng is concerned about wealth inequality. He notes that the AI revolution has caused a concentration of wealth in industries, especially the Internet industry where a winner-takes-most and winner-takes-all dynamic come into play. This flavor is also being spread to other industries.