This page compiles advice offered by researchers and tech professionals regarding different aspects of a career in computer science and artificial intelligence.
iven two people with exactly the same ability, the one person who manages day in and day out to get in one more hour of thinking will be tremendously more productive over a lifetime.
-- Richard Hamming
ive your dreams all you've got and you'll be amazed at the energy that comes out of you.
-- William James
-- Kai-Fu Lee
t's when you're at your lowest point that you have to pick yourself up and continue, go through the discomfort, feel its pain.
How bad do you want it? Answer this question. And start from there.
f you want to know your past, look into your present conditions. If you want to know your future, look into your present actions.
he moment you first encounter a particular book is the right time to read it.
-- Marie Kondō
ost wire walkers, they die when they arrive. They think they have arrived... But they are still on the wire. If you have three steps to do, and you take those steps arrogantly... you think you are invincible... You are going to die.
-- Papa Rudy (The Walk, 2015)
roan" moments = Growth moments -- Moments in life that make you want to run for the hills are the very moments when you should stay put and work through the situation. These are your growth moments. If they feel like “groan” moments, that’s your cue that the situation wants something more from you.
These are opportunities to stretch yourself. These are times to flex your adaptability muscle, which can only work in your favor.
-- Nicole Lipkin
he people who are the most adaptable in life, who can figure out how to work with difficult people, are the ones that people want to be around. When people want to be around you, it enhances your influence.
-- Nicole Lipkin
You have to neglect things if you intend to get what you want done. There's no question about this.
-- Richard Hamming
Machine Learning Research and Engineering Practices
Experiments can take a long time... It is therefore critical to not fall into a slow iteration cycle too early in the course of a short-term project... When analyzing results, be hungry for useful information.
Advice for Short-Term Machine Learning Research Projects
— Tim Rocktäschel, Jakob Foerster and Greg Farquhar (08/29/2018)
Look into the data... Set up end-to-end train/evaluation skeleton... Get dumb baselines... Overfit... Regularize... Tune...
A Recipe for Training Neural Networks
— Andrej Karpathy (04/25/2019)
Backpropagation is a leaky abstraction; it is a credit assignment scheme with non-trivial consequences. If you try to ignore how it works under the hood because “TensorFlow automagically makes my networks learn”, you will not be ready to wrestle with the dangers it presents, and you will be much less effective at building and debugging neural networks.
Yes, You Should Understand Backprop
— Andrej Karpathy (12/19/2016)
A good ML practitioner needs to know how to choose an algorithm for a particular application and how to monitor and respond to feedback obtained from experiments...
If you have time to tune only one hyperparameter, tune the learning rate... on a logarithmic scale.
Deep Learning Practical Methodology
— Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016)
Quite often, you’ll need to come up with your own diagnostics...
If you’re working on one important ML application for months/years, it’s very valuable for you personally to get an intuitive
understanding of what works and what doesn’t work in your problem.
Advice for Applying Machine Learning
— Andrew Ng (2012)
Rapid iteration is also a prime currency in enabling innovation and discovery...
baselines tend to be artificial, tiny, ancient, ignored, and rarely updated to modern best practices or techniques.
When we lose accurate baselines, we lose our ability to accurately measure our progress over time.
Backing off towards simplicity - why baselines need more love
— Smerity (11/03/2017)
Many widely adopted neural NLP algorithms were built on top of the core ideas in traditional NLP models...
Familiarize yourself with machine learning fundamentals...
Be mind-open and keep track of the related problems in other domains such as computer vision and data mining.
Advice for Beginners in Natural Language Processing
— Jiwei Li (03/14/2019)
We build MXNet from scratch... how far we can go as a team is amazing...
TVM is completely new and cross-disciplinary... we need to uncover lots of problems occluded by existing system design -- it is like building a castle on a deserted island...
I started to appreciate the importance of passing knowledge on... Research is never a lone job...
Being able to spot the right thing to do is equally important as the ability of getting things done... It took the same time to do an easy task as to do a challenging one, so why not picking the problem that excites you the most?
Ten Years of Machine Learning Research
— Tianqi Chen (07/18/2019)
Quit while you are ahead... that is true for both research and product impact... the greatest values of a research team is to be able to jump to new ideas that could have much greater leverage than ideas they currently work on...
You can only do that if you are willing to abandon something that's still yielding results, before it's completely exhausted... This "diminishing return" effect happens across all areas in science and engineering.
Managing Industry Research Teams
-- Fernando Pereira (06/2020)
Read enough so that you start developing intuitions, and trust your intuitions...
In the early days of AI people were completely convinced that the representation needed for intelligence are symbolic representations... and that the essense of intelligence was reasoning...
Now there is this completely different view... thought is just a great, big, vector of neural activity...
Heroes of Deep Learning Interview
-- Geoffrey Hinton (08/08/2017)
Careers are long relative to PhDs, and so you should definitely not feel constrained to stay within the confines of whatever area you did your PhD research. Post-PhD, try to put yourself in an environment where you can work on a wide variety of areas, w/ colleagues who have
a wide variety of interests and expertise and then let your own curiosity take you into different areas... As you do this a few times throughout your career, you gradually build a bigger and bigger "tool belt" of techniques and approaches and kinds of expertise.
Changing Research Areas
-- Jeff Dean (11/1/2020)
The world knowledge is growing fast to the point that it is almost impossible to "catch up with the frontier" of a field by following the development of the field step by step.
It's better to immediately start following the cutting-edge works once one grasps the essential knowledge and tools. You will encounter something you don't know while exploring the cutting-edge. It is fine. You can always go back and learn what's needed.
This is learning on demand.
Learning On Demand
-- Vivian Yun-Nung Chen (11/8/2020)
For most startups, the biggest moat is speed and innovation. Same holds for me and you. The faster we iterate and innovate, faster we grow. All of us have the same 24 hours. Make speed and innovation your moat :)
Make speed and innovation your moat
-- Anshuman Singh (12/14/2020)
Calendars convert time to space. They make the finiteness of time apparent. In a way that physical space constraints are apparent...
It is easier to measure how wrong your time estimates are than it is to fix them...
Replanning is part of the plan.
Calendar. Not to-do lists.
— Devi Parikh (04/25/2018)
At this point, my main professional goal is to maximize this portion of my work (solo research), since it's when I make forward progress toward
what I'm most passionate about, and also what matters most for my career advancement...
All of my other work activities are primarily to serve others; this is time for me to grind till my eyes bulge out...
RAMs are one of my main strategies for getting unstuck from local maxima in my daily research grind.
How I've been spending my time as a first-semester assistant professor
— Philip J. Guo (10/2014)
I can no longer sustain my student schedule since the nature of a faculty job involves extreme multitasking...
I'm finally beginning to get the hang of being an academic researcher, but I have a lot less time to execute on my ideas.
So the main challenge moving forward is to find ways to do more real work in fewer hours.
Learning Management: Thoughts for Junior Researchers
— Philip J. Guo (01/2014)
A professor at a research-intensive institution needs to be a researcher, fundraiser, teacher, academic community citizen, department citzen...
A junior professor must both be a maker and a manager... one valuable skill I learned was how to carefully schedule meetings so as to give me
enough maker time each day to march forward on my own research...
Whenever I reply to an email... give... a concrete action item... "fire and forget".
Preparing for Junior Faculty Life
— Philip J. Guo (12/2013)
get good at what you need to do for your job (computer programming and technical writing)... The better you are at your craft, the faster you will be able to do it, and the more you will be willing to try alternatives which might lead to creative innovation...
Figuring out what kind of work is useful in your particular situation is easier said than done; it requires perceptiveness, experience, and a willingness to learn from your past inefficiencies...
*How To Be Effective: Perspective of a PhD Student
— Philip J. Guo (02/2012)
You can't afford everything, but you can afford anything... Prioritizing things outside of work, prioritizing rest and being strategic
about it is important for making more time... is not a fixed resource... Rest a small amount at the right time instead of waiting for too long... Some things are either time generative or energy generative...
Morale is also important to how much energy you have... Preparation is a good substitute for time (esp. in collaborations)...
On Time and Serendipity Management.
— Jean Yang/Philip J. Guo (12/22/2016)
This is another trap that comes to you as you become more senior... there are so many little decisions you have to make... there is a danger of ...
losing your unconcious mind... one of the most important problem solving tools you have...
You need to have strategies for compartmentalizing... that allow you to be a person in the world... without taking so much of your
attention that you're not able to do deep thought.
When you are a researcher, it's the deep long-term thought... that's what you are hired for, that's where your value is at...
have self-knowledge... figure out in what environment, at what time of day am I most creative, and protect that time!
Stress in Research
— Charles Sutton (04/16/2019)
“Faking it” is the antithesis of authentic leadership...
Developing as a leader is hard work...
What stood out for every one of them was how hard they had worked to develop themselves, and the painful lessons they learned from their mistakes and failures. Through those very difficult experiences they developed the self-awareness, confidence, courage and resilience to persevere through the most difficult challenges, and imbue their colleagues with confidence in their leadership and ability to succeed...
Authentic leaders... acknowledge their shortcomings and admit their errors, which enables them to connect with others and inspire teammates.
You Won't Make It If You Fake It
-- Bill George (12/08/2015)
A simplistic understanding of what authenticity means can hinder your growth and limit your impact...
By viewing ourselves as works in progress and evolving our professional identities through trial and error, we can develop a personal style that feels right to us and suits our organizations’ changing needs...
Learn from diverse role models... Work on getting better... Don't stick to "your story".
The Authenticity Paradox
-- Herminia Ibarra (01/2015)
True authenticity doesn't require you to tell anyone else what your values are, because everyone will be able to see your values in the way you live your life. When you are fully authentic you don't reveal your chosen values, you become them – you live them...
If you want to be authentic in the workplace, don't focus on revealing who you are, instead focus on creating and truly becoming yourself.
Authenticity is about being the author of your own story
-- Nina Burrowes (04/11/2014)
A great leadership style can make people appear more competent than they truly are, and a poor style can drag down a superior skill set... Style is distinct from personality... Power and attractiveness, neither set of markers is inherently good or bad...
Lean powerful with more-senior people, and lean attractive when talking to more-junior people... leadership style cannot be fully divorced from unconscious biases and discrimination... Minorities who do not conform to an organization’s dominant culture may be penalized.
How to Develop Your Leadership Style
-- Suzanne J. Peterson , Robin Abramson and R.K. Stutman (11/2020)
When your vision... is what you can do single-handedly, then you should pursue it. The day your vision... is bigger than what you can do single-handedly, then you have to move toward management. And the bigger the vision is, the farther in management you have to go...
You can't make it happen from the bottom very easily.
It depends upon what goals and... desires you have... as they change in life, you have to be prepared to change...
Keep an open mind... But when you do choose a path... be aware of... the choice you have made. Don't try to do both sides.
Comparison Between Research and Management
— Richard Hamming (03/07/1986)
The nature of the choices that people have to make has been changing because there are newer kind of models for dividing your time than... that tended
to exist in the past... Any situation where you are working out the fit with an employer, the core question has to deal with "value"... If it comes down to the incentives and what
it is that supporting the research,
... companies... have a bottomline that is defined in a very different way from the traditional definition of the
bottomline incentives in academia.
Research in Academia versus Industry.
— Philip Resnik and Jason Baldridge (05/31/2019)
We live in a time of hyper-competitiveness... The Career Perspective, the Identity Perspective, the Stability Perspective, and the Variability Perspective... One fundamental difference in academic values is ... a Neat or a Scruffy: None of these roles is inherently more valuable than another — both are needed to make progress in science. The best results in science often come from critical discussion and work across these camps.
How to Pick Your Grad School.
— Tim Dettmers (03/10/2020)
We’ve found that the biggest contributions at OpenAI come from cross-functional experts, so we either need to hire them or grow them here... On a typical learning day, people do things like:
Reimplement papers. Follow deep learning tutorials. Play with new tools in cluster management, compilation, virtual world generation, or coding paradigms. Learn how to do research on bite-sized problems.
Read about new developments in seemingly unrelated areas of AI.
How Learning Day Was Started and How It Works at OpenAI
— OpenAI (08/01/2019)
... Hallway conversations can be even more fruitful. Do everything you can to cultivate such conversations...
You should also tell others about your research. Think about how to frame your work to convey how interesting it is...
Plan your pitch, practice it with your friends, then further refine it through interactions at the conference...
You'll learn a lot from... seeing what confuses people and receiving their ideas and suggestions...
also be sure to ask others about their work (even doing so first)...
Keep an open mind, and try to deeply understand their research.
Attending an Academic Conference
— Michael D. Ernst (07/2004)
Create a process for students to request a letter and make sure they know about it well in advance... Communicate what's needed for a strong letter, what can and cannot be said...
Not only comment on strengths, achievements and potential, but also try to convey why they might fit... try to comment on the individual's traits beyond the prescriptive nature of classroom or lab when you can do so meaningfully...
It is important for cultivating a sense of trust with those who read the letters now and in the future...
Understand how the letter may be read in the context of an entire application package via frank conversations... fill in the gaps... tell detailed stories...
Writing the Perfect Recommendation Letter
-- Nature Career Column by Andy Tay (07/20/2020)
Whenever appropriate, papers on machine learning will... be evaluated on the basis of... 1) Novelty of algorithm; 2) Novelty of application/problem; 3) Difficulty of application; 4) Quality of results; 5) Insight conveyed...
Application papers should...
describe work that has direct relevance to, and addresses the full complexity of, solving a non-trivial problem.
Guidelines for Writing A Good NeurIPS Paper
— The NeurIPS 2006 Program Committee (2013)
Your purpose is to communicate specific ideas, and everything about your paper should contribute to this goal...
you should give away the punchline...
Make your writing crisp and to the point...
Avoid puffery, self-congratulation, and value judgments: give the facts and let the reader judge.
How to Write A Technical Paper
— Michael D. Ernst (11/10/2018)
Be prepared with multiple potential projects... be sure to know what the goal is, what is needed to accomplish it...
Some undergraduates may prefer more direction...
Your primary goal should be to ensure frequent communication... understand the approach and the implementation, at least at some level.
(Graduate Student) Collaborating with Undergraduates in Research
— Michael D. Ernst (01/28/2019)
I tend... to imagine that the person who wrote this paper is very early in their career... and the last thing I want to do is to discourage that person from continuing to try...
What we really want is for the scientific field to move forward...
and for important new information that's going to affect the way people do things to get out there.
On Writing Quality Peer Reviews
— Noah A. Smith (01/07/2019)
Your comments should begin by summarizing the main ideas of the submission and relating these ideas to previous work...
You should then summarize the strengths and weaknesses of the submission, focusing on each of the following four criteria: Quality, Clarity, Originality and Significance.
NeurIPS Reviewer, AC & SAC Guidelines
— The NeurIPS Program Committee (2018)
Personal branding is no substitute for doing high-quality technical work; it’s just the means by which you can share this work with a broader audience...
It’s important to offer useful information to your audience, and not just advertise your product or company...
It can be intimidating to start blogging, but remember that your target audience is you-6-months-ago...
The context of your particular background, your particular style, and your knowledge level will give a different twist to what you’re writing about.
Making Peace with Personal Branding
— Rachel Thomas (12/18/2017)
It's like a resume, only better... Organizing knowledge always helps me synthesize my own ideas...
if I'm asked a question that I think someone else would also be interested in, I try to write it up...
You're best positioned to help people one step behind you.
Why you (yes, you) should blog
— Rachel Thomas (07/28/2017)
You see again and again that it is more than one thing from a good person...
a lot of times there is repetition...
Once he got well started, his shyness, his awkwardness, his inarticulateness, fell away and he became much more productive in many other ways...
great scientists... have courage... will go forward under incredible circumstances...
The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity — it is very much like compound interest...
The great scientists often make this error. They fail to continue to plant the little acorns from which the mighty oak trees grow...
You and Your Research.
— Richard Hamming (03/07/1986)
1. Actively and proactively seek out feedback; 2. absorb and seek out related information; 3. think about edge cases, corner cases, all sorts of possibilities and contingencies to
make the project more robust and more failure-proof.
What People Do When They Really Care About A Project.
— Philip J. Guo (01/07/2020)