This page compiles advice offered by researchers and tech professionals regarding different aspects of a career in computer science and artificial intelligence.
iven two people with exactly the same ability, the one person who manages day in and day out to get in one more hour of thinking will be tremendously more productive over a lifetime.
-- Richard Hamming
Always focus on the big picture.
-- Kai-Fu Lee
t's when you're at your lowest point that you have to pick yourself up and continue, go through the discomfort, feel its pain.
How bad do you want it? Answer this question. And start from there.
f you want to know your past, look into your present conditions. If you want to know your future, look into your present actions.
he moment you first encounter a particular book is the right time to read it.
-- Marie Kondō
ost wire walkers, they die when they arrive. They think they have arrived... But they are still on the wire. If you have three steps to do, and you take those steps arrogantly... you think you are invincible... You are going to die.
-- Papa Rudy (The Walk, 2015)
roan" moments = Growth moments -- Moments in life that make you want to run for the hills are the very moments when you should stay put and work through the situation. These are your growth moments. If they feel like “groan” moments, that’s your cue that the situation wants something more from you.
These are opportunities to stretch yourself. These are times to flex your adaptability muscle, which can only work in your favor.
-- Nicole Lipkin
he people who are the most adaptable in life, who can figure out how to work with difficult people, are the ones that people want to be around. When people want to be around you, it enhances your influence.
-- Nicole Lipkin
You have to neglect things if you intend to get what you want done. There's no question about this.
-- Richard Hamming
ive your dreams all you've got and you'll be amazed at the energy that comes out of you.
-- William James
he Struggle has no mercy. The Struggle is not failure; but it causes failure. Especially if you are weak
. Always if you are weak..
-- Ben Horowitz
f you focus on the wall, you will drive right into it. If you focus on the road, you will follow the road. Focus on where you are going rather than on what you hope to avoid.
-- Ben Horowitz
f you want to work and continue to work, you must continually dedicate yourself to retaining your individual competitive advantage.
-- Andrew S. Grove
Machine Learning Research and Engineering Practices
Experiments can take a long time... It is therefore critical to not fall into a slow iteration cycle too early in the course of a short-term project... When analyzing results, be hungry for useful information.
Advice for Short-Term Machine Learning Research Projects
— Tim Rocktäschel, Jakob Foerster and Greg Farquhar (08/29/2018)
Look into the data... Set up end-to-end train/evaluation skeleton... Get dumb baselines... Overfit... Regularize... Tune...
A Recipe for Training Neural Networks
— Andrej Karpathy (04/25/2019)
Backpropagation is a leaky abstraction; it is a credit assignment scheme with non-trivial consequences. If you try to ignore how it works under the hood because “TensorFlow automagically makes my networks learn”, you will not be ready to wrestle with the dangers it presents, and you will be much less effective at building and debugging neural networks.
Yes, You Should Understand Backprop
— Andrej Karpathy (12/19/2016)
A good ML practitioner needs to know how to choose an algorithm for a particular application and how to monitor and respond to feedback obtained from experiments...
If you have time to tune only one hyperparameter, tune the learning rate... on a logarithmic scale.
Deep Learning Practical Methodology
— Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016)
Quite often, you’ll need to come up with your own diagnostics...
If you’re working on one important ML application for months/years, it’s very valuable for you personally to get an intuitive
understanding of what works and what doesn’t work in your problem.
Advice for Applying Machine Learning
— Andrew Ng (2012)
Rapid iteration is also a prime currency in enabling innovation and discovery...
baselines tend to be artificial, tiny, ancient, ignored, and rarely updated to modern best practices or techniques.
When we lose accurate baselines, we lose our ability to accurately measure our progress over time.
Backing off towards simplicity - why baselines need more love
— Smerity (11/03/2017)
The problem with this sort of low-key fraud is that it’s insidious, it’s subtle...
For the first time, researchers reading conference proceedings will be forced to wonder: does this work truly merit my attention? Or is its publication simply the result of fraud...
Together, we can force the community to reckon with its own shortcomings, and develop stronger, better, and more scientific norms.
"Please Commit More Blatant Academic Fraud"
— Jacob Buckman (05/29/2021)
Many widely adopted neural NLP algorithms were built on top of the core ideas in traditional NLP models...
Familiarize yourself with machine learning fundamentals...
Be mind-open and keep track of the related problems in other domains such as computer vision and data mining.
Advice for Beginners in Natural Language Processing
— Jiwei Li (03/14/2019)
We build MXNet from scratch... how far we can go as a team is amazing...
TVM is completely new and cross-disciplinary... we need to uncover lots of problems occluded by existing system design -- it is like building a castle on a deserted island...
I started to appreciate the importance of passing knowledge on... Research is never a lone job...
Being able to spot the right thing to do is equally important as the ability of getting things done... It took the same time to do an easy task as to do a challenging one, so why not picking the problem that excites you the most?
Ten Years of Machine Learning Research
— Tianqi Chen (07/18/2019)
Immerse yourself into the problem you're trying to solve... Read the few most important papers in a field in depth, this enables you to "predict" the ideas in more papers, and you can skim the rest of the papers... Grasp the main direction...
In writing, try to present an idea from the broadest perspective... A work that proposes a new framework can be more impactful than a work that adds addiitonal features...
Summary of My PhD Research
— Yuandong Tian (11/01/2017)
Quit while you are ahead... that is true for both research and product impact... the greatest values of a research team is to be able to jump to new ideas that could have much greater leverage than ideas they currently work on...
You can only do that if you are willing to abandon something that's still yielding results, before it's completely exhausted... This "diminishing return" effect happens across all areas in science and engineering.
Managing Industry Research Teams
-- Fernando Pereira (06/2020)
Read enough so that you start developing intuitions, and trust your intuitions...
In the early days of AI people were completely convinced that the representation needed for intelligence are symbolic representations... and that the essense of intelligence was reasoning...
Now there is this completely different view... thought is just a great, big, vector of neural activity...
Heroes of Deep Learning Interview
-- Geoffrey Hinton (08/08/2017)
Careers are long relative to PhDs, and so you should definitely not feel constrained to stay within the confines of whatever area you did your PhD research. Post-PhD, try to put yourself in an environment where you can work on a wide variety of areas, w/ colleagues who have
a wide variety of interests and expertise and then let your own curiosity take you into different areas... As you do this a few times throughout your career, you gradually build a bigger and bigger "tool belt" of techniques and approaches and kinds of expertise.
Changing Research Areas
-- Jeff Dean (11/1/2020)
The world knowledge is growing fast to the point that it is almost impossible to "catch up with the frontier" of a field by following the development of the field step by step.
It's better to immediately start following the cutting-edge works once one grasps the essential knowledge and tools. You will encounter something you don't know while exploring the cutting-edge. It is fine. You can always go back and learn what's needed.
This is learning on demand.
Learning On Demand
-- Vivian Yun-Nung Chen (11/8/2020)
For most startups, the biggest moat is speed and innovation. Same holds for me and you. The faster we iterate and innovate, faster we grow. All of us have the same 24 hours. Make speed and innovation your moat :)
Make speed and innovation your moat
-- Anshuman Singh (12/14/2020)
Set a deadline for yourself and just start exploring solutions. You'll quickly learn more as you start solving the problem, and that will lead you to iterate into a better solution...
If you don't have a good grasp of the universe of what's possible, you can't design a good system...
Worry less about elegance and perfection; instead strive for continuous improvement and creating a livable system that your team enjoys working with and sustainably delivers value...
Don't mistake humility for ignorance.
20 Things I've Learned in My 20 Years as A Software Engineer
-- Justin Etheredge (10/2021)
This short paper introduces "the three-pass approach" for reading a scientific paper: build the bird's-eye view (first pass) first and then try to grasp the main content ignoring details such as proofs (second pass);
in the third pass, attempt to deeply understand and virtually re-implement the paper, identify its strong and weak points, pinpoint implicit assumptions and missing citations etc.
A three-step approach was also presented for doing literature survey: finding key papers using search engine, identify key papers and researchers in the area via shared citations, and identify top conferences and look through their recent proceedings.
How to Read a Paper
— Srinivasan Keshav
It is interesting that most articles about (scientific) paper reading converge on a few shared ideas: reading with a purpose, starting with the abstract and conclusion section first, taking shortcuts, using a reference manager, building a mental paper system and slotting the paper you're reading into the system...
This article is a quite comprehensive compilation of paper reading tips offered by researchers with different backgrounds.
How to (seriously) read a scientific paper
— Elisabeth Pain (03/21/2016)
Set aside time to read... Organize papers and notes... Before you start, make a plan... Approach each paper section with different goals and questions in mind.
How to Read Scientific Papers
— John Leonard (07/26/2020)
Read strategically, not linearly... Be purposeful... Have a critical perspective... Take Notes.
How to Read for Grad School
— Miriam E. Sweeney (06/02/2012)
A note of Mortimer Adler's living classic "How to Read a Book". Interestingly but unsurprisingly, the tips are close parallel of those offered by the articles above on scientific paper reading.
The author characterize reading in four levels: elementary reading, inspectional reading, analytical reading and syntopical reading.
The central idea is to use the right reading technique depending on why you're reading. Be demanding, ask the right question in the right order, and structure your reading based on those goals.
How to Read a Book: The Ultimate Guide by Mortimer Adler
— Farnam Street
To get the most out of each book we read, it is vital we know how to record, reflect on, and put into action our conclusions...
Active reading requires focus...
When you come across an important passage or concept, pause and visualize it. Make the picture as salient and distinctive as possible by connecting it to other ideas already in your brain...
Apply what you've learned...
Skim a lot of books. Read a few. Immediately re-read the best ones twice.
How to Remember What You Read
— Farnam Street (08/2021)
Whenever appropriate, papers on machine learning will... be evaluated on the basis of... 1) Novelty of algorithm; 2) Novelty of application/problem; 3) Difficulty of application; 4) Quality of results; 5) Insight conveyed...
Application papers should...
describe work that has direct relevance to, and addresses the full complexity of, solving a non-trivial problem.
Guidelines for Writing A Good NeurIPS Paper
— The NeurIPS 2006 Program Committee (2013)
Your purpose is to communicate specific ideas, and everything about your paper should contribute to this goal...
you should give away the punchline...
Make your writing crisp and to the point...
Avoid puffery, self-congratulation, and value judgments: give the facts and let the reader judge.
How to Write A Technical Paper
— Michael D. Ernst (11/10/2018)
Calendars convert time to space. They make the finiteness of time apparent. In a way that physical space constraints are apparent...
It is easier to measure how wrong your time estimates are than it is to fix them...
Replanning is part of the plan.
Calendar. Not to-do lists.
— Devi Parikh (04/25/2018)
At this point, my main professional goal is to maximize this portion of my work (solo research), since it's when I make forward progress toward
what I'm most passionate about, and also what matters most for my career advancement...
All of my other work activities are primarily to serve others; this is time for me to grind till my eyes bulge out...
RAMs are one of my main strategies for getting unstuck from local maxima in my daily research grind.
How I've been spending my time as a first-semester assistant professor
— Philip J. Guo (10/2014)
I can no longer sustain my student schedule since the nature of a faculty job involves extreme multitasking...
I'm finally beginning to get the hang of being an academic researcher, but I have a lot less time to execute on my ideas.
So the main challenge moving forward is to find ways to do more real work in fewer hours.
Learning Management: Thoughts for Junior Researchers
— Philip J. Guo (01/2014)
A professor at a research-intensive institution needs to be a researcher, fundraiser, teacher, academic community citizen, department citzen...
A junior professor must both be a maker and a manager... one valuable skill I learned was how to carefully schedule meetings so as to give me
enough maker time each day to march forward on my own research...
Whenever I reply to an email... give... a concrete action item... "fire and forget".
Preparing for Junior Faculty Life
— Philip J. Guo (12/2013)
get good at what you need to do for your job (computer programming and technical writing)... The better you are at your craft, the faster you will be able to do it, and the more you will be willing to try alternatives which might lead to creative innovation...
Figuring out what kind of work is useful in your particular situation is easier said than done; it requires perceptiveness, experience, and a willingness to learn from your past inefficiencies...
*How To Be Effective: Perspective of a PhD Student
— Philip J. Guo (02/2012)
You can't afford everything, but you can afford anything... Prioritizing things outside of work, prioritizing rest and being strategic
about it is important for making more time... is not a fixed resource... Rest a small amount at the right time instead of waiting for too long... Some things are either time generative or energy generative...
Morale is also important to how much energy you have... Preparation is a good substitute for time (esp. in collaborations)...
On Time and Serendipity Management.
— Jean Yang/Philip J. Guo (12/22/2016)
This is another trap that comes to you as you become more senior... there are so many little decisions you have to make... there is a danger of ...
losing your unconcious mind... one of the most important problem solving tools you have...
You need to have strategies for compartmentalizing... that allow you to be a person in the world... without taking so much of your
attention that you're not able to do deep thought.
When you are a researcher, it's the deep long-term thought... that's what you are hired for, that's where your value is at...
have self-knowledge... figure out in what environment, at what time of day am I most creative, and protect that time!
Stress in Research
— Charles Sutton (04/16/2019)
“Faking it” is the antithesis of authentic leadership...
Developing as a leader is hard work...
What stood out for every one of them was how hard they had worked to develop themselves, and the painful lessons they learned from their mistakes and failures. Through those very difficult experiences they developed the self-awareness, confidence, courage and resilience to persevere through the most difficult challenges, and imbue their colleagues with confidence in their leadership and ability to succeed...
Authentic leaders... acknowledge their shortcomings and admit their errors, which enables them to connect with others and inspire teammates.
You Won't Make It If You Fake It
-- Bill George (12/08/2015)
A simplistic understanding of what authenticity means can hinder your growth and limit your impact...
By viewing ourselves as works in progress and evolving our professional identities through trial and error, we can develop a personal style that feels right to us and suits our organizations’ changing needs...
Learn from diverse role models... Work on getting better... Don't stick to "your story".
The Authenticity Paradox
-- Herminia Ibarra (01/2015)
True authenticity doesn't require you to tell anyone else what your values are, because everyone will be able to see your values in the way you live your life. When you are fully authentic you don't reveal your chosen values, you become them – you live them...
If you want to be authentic in the workplace, don't focus on revealing who you are, instead focus on creating and truly becoming yourself.
Authenticity is about being the author of your own story
-- Nina Burrowes (04/11/2014)
A great leadership style can make people appear more competent than they truly are, and a poor style can drag down a superior skill set... Style is distinct from personality... Power and attractiveness, neither set of markers is inherently good or bad...
Lean powerful with more-senior people, and lean attractive when talking to more-junior people... leadership style cannot be fully divorced from unconscious biases and discrimination... Minorities who do not conform to an organization’s dominant culture may be penalized.
How to Develop Your Leadership Style
-- Suzanne J. Peterson , Robin Abramson and R.K. Stutman (11/2020)
When your vision... is what you can do single-handedly, then you should pursue it. The day your vision... is bigger than what you can do single-handedly, then you have to move toward management. And the bigger the vision is, the farther in management you have to go...
You can't make it happen from the bottom very easily.
It depends upon what goals and... desires you have... as they change in life, you have to be prepared to change...
Keep an open mind... But when you do choose a path... be aware of... the choice you have made. Don't try to do both sides.
Comparison Between Research and Management
— Richard Hamming (03/07/1986)
The nature of the choices that people have to make has been changing because there are newer kind of models for dividing your time than... that tended
to exist in the past... Any situation where you are working out the fit with an employer, the core question has to deal with "value"... If it comes down to the incentives and what
it is that supporting the research,
... companies... have a bottomline that is defined in a very different way from the traditional definition of the
bottomline incentives in academia.
Research in Academia versus Industry.
— Philip Resnik and Jason Baldridge (05/31/2019)
We live in a time of hyper-competitiveness... The Career Perspective, the Identity Perspective, the Stability Perspective, and the Variability Perspective... One fundamental difference in academic values is ... a Neat or a Scruffy: None of these roles is inherently more valuable than another — both are needed to make progress in science. The best results in science often come from critical discussion and work across these camps.
How to Pick Your Grad School.
— Tim Dettmers (03/10/2020)
Good work doesn’t speak for itself... Convey why are you spending time solving this problem, and why NOW... Regardless of the approach, the key is to speak to the impact and outcomes...
Those who cares about your work are your stakeholders... you need to communicate with them on a regular basis... it's not enough for you to speak for your good work alone.
My First Career Lesson.
— Catherine Ye (05/29/2021)
The value of anything you build lies in its impact on the society... Exceling in one thing is better than being good on a few things... Save enough time and energy for communication...
For on the impactful things... You need to understand well the impact, both for the very moment and for the future, and organize resources to excel on the most impactful things.
Reflection on the first five years of my career.
— Mu Li (05/23/2021)
We’ve found that the biggest contributions at OpenAI come from cross-functional experts, so we either need to hire them or grow them here... On a typical learning day, people do things like:
Reimplement papers. Follow deep learning tutorials. Play with new tools in cluster management, compilation, virtual world generation, or coding paradigms. Learn how to do research on bite-sized problems.
Read about new developments in seemingly unrelated areas of AI.
How Learning Day Was Started and How It Works at OpenAI
— OpenAI (08/01/2019)
I tend... to imagine that the person who wrote this paper is very early in their career... and the last thing I want to do is to discourage that person from continuing to try...
What we really want is for the scientific field to move forward...
and for important new information that's going to affect the way people do things to get out there.
On Writing Quality Peer Reviews
— Noah A. Smith (01/07/2019)
Your comments should begin by summarizing the main ideas of the submission and relating these ideas to previous work...
You should then summarize the strengths and weaknesses of the submission, focusing on each of the following four criteria: Quality, Clarity, Originality and Significance.
NeurIPS Reviewer, AC & SAC Guidelines
— The NeurIPS Program Committee (2018)
Our current publication system should be redesigned to maximize the rate of progress in our field. This means accelerating the speed at which new ideas and results are exchanged, disseminated, and evaluated...
Ideas that turn out to be highly influential are sometimes held up for months (if not years) in reviewing purgatory, particularly if they require several years to come to maturity... The friction in our publication system is slowing the progress of our field. It makes progress incremental. And it makes our conferences somewhat boring...
A New Publishing Model in Computer Science
— Yann Lecun (2011)
... Hallway conversations can be even more fruitful. Do everything you can to cultivate such conversations...
You should also tell others about your research. Think about how to frame your work to convey how interesting it is...
Plan your pitch, practice it with your friends, then further refine it through interactions at the conference...
You'll learn a lot from... seeing what confuses people and receiving their ideas and suggestions...
also be sure to ask others about their work (even doing so first)...
Keep an open mind, and try to deeply understand their research.
Attending an Academic Conference
— Michael D. Ernst (07/2004)
Be prepared with multiple potential projects... be sure to know what the goal is, what is needed to accomplish it...
Some undergraduates may prefer more direction...
Your primary goal should be to ensure frequent communication... understand the approach and the implementation, at least at some level.
(Graduate Student) Collaborating with Undergraduates in Research
— Michael D. Ernst (01/28/2019)
Create a process for students to request a letter and make sure they know about it well in advance... Communicate what's needed for a strong letter, what can and cannot be said...
Not only comment on strengths, achievements and potential, but also try to convey why they might fit... try to comment on the individual's traits beyond the prescriptive nature of classroom or lab when you can do so meaningfully...
It is important for cultivating a sense of trust with those who read the letters now and in the future...
Understand how the letter may be read in the context of an entire application package via frank conversations... fill in the gaps... tell detailed stories...
Writing the Perfect Recommendation Letter
-- Nature Career Column by Andy Tay (07/20/2020)
Personal branding is no substitute for doing high-quality technical work; it’s just the means by which you can share this work with a broader audience...
It’s important to offer useful information to your audience, and not just advertise your product or company...
It can be intimidating to start blogging, but remember that your target audience is you-6-months-ago...
The context of your particular background, your particular style, and your knowledge level will give a different twist to what you’re writing about.
Making Peace with Personal Branding
— Rachel Thomas (12/18/2017)
It's like a resume, only better... Organizing knowledge always helps me synthesize my own ideas...
if I'm asked a question that I think someone else would also be interested in, I try to write it up...
You're best positioned to help people one step behind you.
Why you (yes, you) should blog
— Rachel Thomas (07/28/2017)
You see again and again that it is more than one thing from a good person...
a lot of times there is repetition...
Once he got well started, his shyness, his awkwardness, his inarticulateness, fell away and he became much more productive in many other ways...
great scientists... have courage... will go forward under incredible circumstances...
The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity — it is very much like compound interest...
The great scientists often make this error. They fail to continue to plant the little acorns from which the mighty oak trees grow...
You and Your Research.
— Richard Hamming (03/07/1986)
1. Actively and proactively seek out feedback; 2. absorb and seek out related information; 3. think about edge cases, corner cases, all sorts of possibilities and contingencies to
make the project more robust and more failure-proof.
What People Do When They Really Care About A Project.
— Philip J. Guo (01/07/2020)