Quit while you are ahead... that is true for both research and product impact... the greatest values of a research team is to be able to jump to new ideas that could have much greater leverage than ideas they currently work on... You can only do that if you are willing to abandon something that's still yielding results, before it's completely exhausted... This "diminishing return" effect happens across all areas in science and engineering.

Managing Industry Research Teams -- Fernando Pereira (06/2020)

Read enough so that you start developing intuitions, and trust your intuitions... In the early days of AI people were completely convinced that the representation needed for intelligence are symbolic representations... and that the essense of intelligence was reasoning... Now there is this completely different view... thought is just a great, big, vector of neural activity...

Heroes of Deep Learning Interview -- Geoffrey Hinton (08/08/2017)

Careers are long relative to PhDs, and so you should definitely not feel constrained to stay within the confines of whatever area you did your PhD research. Post-PhD, try to put yourself in an environment where you can work on a wide variety of areas, w/ colleagues who have a wide variety of interests and expertise and then let your own curiosity take you into different areas... As you do this a few times throughout your career, you gradually build a bigger and bigger "tool belt" of techniques and approaches and kinds of expertise.

Changing Research Areas -- Jeff Dean (11/1/2020)

The world knowledge is growing fast to the point that it is almost impossible to "catch up with the frontier" of a field by following the development of the field step by step. It's better to immediately start following the cutting-edge works once one grasps the essential knowledge and tools. You will encounter something you don't know while exploring the cutting-edge. It is fine. You can always go back and learn what's needed. This is learning on demand.

Learning On Demand -- Vivian Yun-Nung Chen (11/8/2020)

... the right problem at the right time in the right way is what counts... But it's easy, there's a million races being run, you just got to get in one of them... When you are famous it is hard to work on small problems... The great scientists often make this error. They try to get the big thing right off. And that isn't the way things go... What appears to be a fault, often, by a change of viewpoint, turns out to be one of the greatest assets you can have... often the great scientists, by turning the problem around a bit, changed a defect to an asset.

You and Your Research. — Richard Hamming (03/07/1986)

“Faking it” is the antithesis of authentic leadership... Developing as a leader is hard work... What stood out for every one of them was how hard they had worked to develop themselves, and the painful lessons they learned from their mistakes and failures. Through those very difficult experiences they developed the self-awareness, confidence, courage and resilience to persevere through the most difficult challenges, and imbue their colleagues with confidence in their leadership and ability to succeed... Authentic leaders... acknowledge their shortcomings and admit their errors, which enables them to connect with others and inspire teammates.

You Won't Make It If You Fake It -- Bill George (12/08/2015)

A simplistic understanding of what authenticity means can hinder your growth and limit your impact... By viewing ourselves as works in progress and evolving our professional identities through trial and error, we can develop a personal style that feels right to us and suits our organizations’ changing needs... Learn from diverse role models... Work on getting better... Don't stick to "your story".

The Authenticity Paradox -- Herminia Ibarra (01/2015)

True authenticity doesn't require you to tell anyone else what your values are, because everyone will be able to see your values in the way you live your life. When you are fully authentic you don't reveal your chosen values, you become them – you live them... If you want to be authentic in the workplace, don't focus on revealing who you are, instead focus on creating and truly becoming yourself.

Authenticity is about being the author of your own story -- Nina Burrowes (04/11/2014)

A great leadership style can make people appear more competent than they truly are, and a poor style can drag down a superior skill set... Style is distinct from personality... Power and attractiveness, neither set of markers is inherently good or bad... Lean powerful with more-senior people, and lean attractive when talking to more-junior people... leadership style cannot be fully divorced from unconscious biases and discrimination... Minorities who do not conform to an organization’s dominant culture may be penalized.

How to Develop Your Leadership Style -- Suzanne J. Peterson , Robin Abramson and R.K. Stutman (11/2020)

I have to get you to drop modesty and say to yourself, "Yes, I would like to do first-class work."... One of the characteristics of successful scientists is having courage. Once you get your courage up and believe that you can do important problems, then you can... Luck favors the prepared mind.... You see again and again that it is more than one thing from a good person... a lot of times there is repetition... usually when they were young they had independent thoughts and had the courage to pursue them... The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity — it is very much like compound interest... Once he got well started, his shyness, his awkwardness, his inarticulateness, fell away and he became much more productive in many other ways...

You and Your Research. — Richard Hamming (03/07/1986)

For most startups, the biggest moat is speed and innovation. Same holds for me and you. The faster we iterate and innovate, faster we grow. All of us have the same 24 hours. Make speed and innovation your moat :)

Make speed and innovation your moat -- Anshuman Singh (12/14/2020)

1. Actively and proactively seek out feedback; 2. absorb and seek out related information; 3. think about edge cases, corner cases, all sorts of possibilities and contingencies to make the project more robust and more failure-proof.

What People Do When They Really Care About A Project. — Philip J. Guo (01/07/2020)

  At this point, my main professional goal is to maximize this portion of my work (solo research), since it's when I make forward progress toward what I'm most passionate about, and also what matters most for my career advancement... All of my other work activities are primarily to serve others; this is time for me to grind till my eyes bulge out... RAMs are one of my main strategies for getting unstuck from local maxima in my daily research grind.

How I've been spending my time as a first-semester assistant professor — Philip J. Guo (10/2014)

  I can no longer sustain my student schedule since the nature of a faculty job involves extreme multitasking... I'm finally beginning to get the hang of being an academic researcher, but I have a lot less time to execute on my ideas. So the main challenge moving forward is to find ways to do more real work in fewer hours.

Learning Management: Thoughts for Junior Researchers — Philip J. Guo (01/2014)

  A professor at a research-intensive institution needs to be a researcher, fundraiser, teacher, academic community citizen, department citzen... A junior professor must both be a maker and a manager... one valuable skill I learned was how to carefully schedule meetings so as to give me enough maker time each day to march forward on my own research... Whenever I reply to an email... give... a concrete action item... "fire and forget".

Preparing for Junior Faculty Life — Philip J. Guo (12/2013)

  DO LESS... get good at what you need to do for your job (computer programming and technical writing)... The better you are at your craft, the faster you will be able to do it, and the more you will be willing to try alternatives which might lead to creative innovation... Figuring out what kind of work is useful in your particular situation is easier said than done; it requires perceptiveness, experience, and a willingness to learn from your past inefficiencies...

^*How To Be Effective: Perspective of a PhD Student — Philip J. Guo (02/2012)

  You can't afford everything, but you can afford anything... Prioritizing things outside of work, prioritizing rest and being strategic about it is important for making more time... is not a fixed resource... Rest a small amount at the right time instead of waiting for too long... Some things are either time generative or energy generative... Morale is also important to how much energy you have... Preparation is a good substitute for time (esp. in collaborations)...

On Time and Serendipity Management. — Jean Yang/Philip J. Guo (12/22/2016)

  Calendars convert time to space. They make the finiteness of time apparent. In a way that physical space constraints are apparent... It is easier to measure how wrong your time estimates are than it is to fix them... Replanning is part of the plan.

Calendar. Not to-do lists. — Devi Parikh (04/25/2018)

  This is another trap that comes to you as you become more senior... there are so many little decisions you have to make... there is a danger of ... losing your unconcious mind... one of the most important problem solving tools you have... You need to have strategies for compartmentalizing... that allow you to be a person in the world... without taking so much of your attention that you're not able to do deep thought. When you are a researcher, it's the deep long-term thought... that's what you are hired for, that's where your value is at... have self-knowledge... figure out in what environment, at what time of day am I most creative, and protect that time!

Stress in Research — Charles Sutton (04/16/2019)

When your vision... is what you can do single-handedly, then you should pursue it. The day your vision... is bigger than what you can do single-handedly, then you have to move toward management. And the bigger the vision is, the farther in management you have to go... You can't make it happen from the bottom very easily. It depends upon what goals and... desires you have... as they change in life, you have to be prepared to change... Keep an open mind... But when you do choose a path... be aware of... the choice you have made. Don't try to do both sides.

Comparison Between Research and Management — Richard Hamming (03/07/1986)

The nature of the choices that people have to make has been changing because there are newer kind of models for dividing your time than... that tended to exist in the past... Any situation where you are working out the fit with an employer, the core question has to deal with "value"... If it comes down to the incentives and what it is that supporting the research, ... companies... have a bottomline that is defined in a very different way from the traditional definition of the bottomline incentives in academia.

Research in Academia versus Industry. — Philip Resnik and Jason Baldridge (05/31/2019)

We live in a time of hyper-competitiveness... The Career Perspective, the Identity Perspective, the Stability Perspective, and the Variability Perspective... One fundamental difference in academic values is ... a Neat or a Scruffy: None of these roles is inherently more valuable than another — both are needed to make progress in science. The best results in science often come from critical discussion and work across these camps.

How to Pick Your Grad School. — Tim Dettmers (03/10/2020)

Good work doesn’t speak for itself... Convey why are you spending time solving this problem, and why NOW... Regardless of the approach, the key is to speak to the impact and outcomes... Those who cares about your work are your stakeholders... you need to communicate with them on a regular basis... it's not enough for you to speak for your good work alone.

My First Career Lesson. — Catherine Ye (05/29/2021)

The value of anything you build lies in its impact on the society... Exceling in one thing is better than being good on a few things... Save enough time and energy for communication... For on the impactful things... You need to understand well the impact, both for the very moment and for the future, and organize resources to excel on the most impactful things.

Reflection on the first five years of my career. — Mu Li (05/23/2021)

  Personal branding is no substitute for doing high-quality technical work; it’s just the means by which you can share this work with a broader audience... It’s important to offer useful information to your audience, and not just advertise your product or company... It can be intimidating to start blogging, but remember that your target audience is you-6-months-ago... The context of your particular background, your particular style, and your knowledge level will give a different twist to what you’re writing about.

Making Peace with Personal Branding — Rachel Thomas (12/18/2017)

  It's like a resume, only better... Organizing knowledge always helps me synthesize my own ideas... if I'm asked a question that I think someone else would also be interested in, I try to write it up... You're best positioned to help people one step behind you.

Why you (yes, you) should blog — Rachel Thomas (07/28/2017)

  To change a field or create a new one, you have to bring people along with you. That is, one has to become an influencer... Every scientist is an influencer. The most successful ones are not just creative but also master the technology of their time to get their ideas across.

Victoria: "Promotion" matters for science. How can one "sell" their ideas without "pushing" the audience? Prof. Black argues in his blog post that the key is to "teach" the audience your new idea and its connection to previous ideas. This is what readers want to know and sharing them creates value, support and excitement. Black also argues that scientists should be the drivers of communication, and maintain constant conversation with their readers.

Scientific communication in the age of influencers — Michael Black (01/28/2024)

  ... Hallway conversations can be even more fruitful. Do everything you can to cultivate such conversations... You should also tell others about your research. Think about how to frame your work to convey how interesting it is... Plan your pitch, practice it with your friends, then further refine it through interactions at the conference... You'll learn a lot from... seeing what confuses people and receiving their ideas and suggestions... also be sure to ask others about their work (even doing so first)... Keep an open mind, and try to deeply understand their research.

Attending an Academic Conference — Michael D. Ernst (07/2004)

  Experiments can take a long time... It is therefore critical to not fall into a slow iteration cycle too early in the course of a short-term project... When analyzing results, be hungry for useful information.

Advice for Short-Term Machine Learning Research Projects — Tim Rocktäschel, Jakob Foerster and Greg Farquhar (08/29/2018)

  Look into the data... Set up end-to-end train/evaluation skeleton... Get dumb baselines... Overfit... Regularize... Tune...

A Recipe for Training Neural Networks — Andrej Karpathy (04/25/2019)

  Backpropagation is a leaky abstraction; it is a credit assignment scheme with non-trivial consequences. If you try to ignore how it works under the hood because “TensorFlow automagically makes my networks learn”, you will not be ready to wrestle with the dangers it presents, and you will be much less effective at building and debugging neural networks.

Yes, You Should Understand Backprop — Andrej Karpathy (12/19/2016)

  A good ML practitioner needs to know how to choose an algorithm for a particular application and how to monitor and respond to feedback obtained from experiments... If you have time to tune only one hyperparameter, tune the learning rate... on a logarithmic scale.

Deep Learning Practical Methodology — Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016)

    Quite often, you’ll need to come up with your own diagnostics... If you’re working on one important ML application for months/years, it’s very valuable for you personally to get an intuitive understanding of what works and what doesn’t work in your problem.

Advice for Applying Machine Learning — Andrew Ng (2012)

  Rapid iteration is also a prime currency in enabling innovation and discovery... baselines tend to be artificial, tiny, ancient, ignored, and rarely updated to modern best practices or techniques. When we lose accurate baselines, we lose our ability to accurately measure our progress over time.

Backing off towards simplicity - why baselines need more love — Smerity (11/03/2017)

  The problem with this sort of low-key fraud is that it’s insidious, it’s subtle... For the first time, researchers reading conference proceedings will be forced to wonder: does this work truly merit my attention? Or is its publication simply the result of fraud... Together, we can force the community to reckon with its own shortcomings, and develop stronger, better, and more scientific norms.

"Please Commit More Blatant Academic Fraud" — Jacob Buckman (05/29/2021)

  Many widely adopted neural NLP algorithms were built on top of the core ideas in traditional NLP models... Familiarize yourself with machine learning fundamentals... Be mind-open and keep track of the related problems in other domains such as computer vision and data mining.

Advice for Beginners in Natural Language Processing — Jiwei Li (03/14/2019)

  We build MXNet from scratch... how far we can go as a team is amazing... TVM is completely new and cross-disciplinary... we need to uncover lots of problems occluded by existing system design -- it is like building a castle on a deserted island... I started to appreciate the importance of passing knowledge on... Research is never a lone job... Being able to spot the right thing to do is equally important as the ability of getting things done... It took the same time to do an easy task as to do a challenging one, so why not picking the problem that excites you the most?

Ten Years of Machine Learning Research — Tianqi Chen (07/18/2019)

  Immerse yourself into the problem you're trying to solve... Read the few most important papers in a field in depth, this enables you to "predict" the ideas in more papers, and you can skim the rest of the papers... Grasp the main direction... In writing, try to present an idea from the broadest perspective... A work that proposes a new framework can be more impactful than a work that adds addiitonal features...

Summary of My PhD Research — Yuandong Tian (11/01/2017)

Set a deadline for yourself and just start exploring solutions. You'll quickly learn more as you start solving the problem, and that will lead you to iterate into a better solution... If you don't have a good grasp of the universe of what's possible, you can't design a good system... Worry less about elegance and perfection; instead strive for continuous improvement and creating a livable system that your team enjoys working with and sustainably delivers value... Don't mistake humility for ignorance.

20 Things I've Learned in My 20 Years as A Software Engineer -- Justin Etheredge (10/2021)

  This short paper introduces "the three-pass approach" for reading a scientific paper: build the bird's-eye view (first pass) first and then try to grasp the main content ignoring details such as proofs (second pass); in the third pass, attempt to deeply understand and virtually re-implement the paper, identify its strong and weak points, pinpoint implicit assumptions and missing citations etc. A three-step approach was also presented for doing literature survey: finding key papers using search engine, identify key papers and researchers in the area via shared citations, and identify top conferences and look through their recent proceedings.

How to Read a Paper — Srinivasan Keshav

  It is interesting that most articles about (scientific) paper reading converge on a few shared ideas: reading with a purpose, starting with the abstract and conclusion section first, taking shortcuts, using a reference manager, building a mental paper system and slotting the paper you're reading into the system... This article is a quite comprehensive compilation of paper reading tips offered by researchers with different backgrounds.

How to (seriously) read a scientific paper — Elisabeth Pain (03/21/2016)

  Set aside time to read... Organize papers and notes... Before you start, make a plan... Approach each paper section with different goals and questions in mind.

How to Read Scientific Papers — John Leonard (07/26/2020)

  Read strategically, not linearly... Be purposeful... Have a critical perspective... Take Notes.

How to Read for Grad School — Miriam E. Sweeney (06/02/2012)

  A note of Mortimer Adler's living classic "How to Read a Book". Interestingly but unsurprisingly, the tips are close parallel of those offered by the articles above on scientific paper reading. The author characterize reading in four levels: elementary reading, inspectional reading, analytical reading and syntopical reading. The central idea is to use the right reading technique depending on why you're reading. Be demanding, ask the right question in the right order, and structure your reading based on those goals.

How to Read a Book: The Ultimate Guide by Mortimer Adler — Farnam Street

  To get the most out of each book we read, it is vital we know how to record, reflect on, and put into action our conclusions... Active reading requires focus... When you come across an important passage or concept, pause and visualize it. Make the picture as salient and distinctive as possible by connecting it to other ideas already in your brain... Apply what you've learned... Skim a lot of books. Read a few. Immediately re-read the best ones twice.

How to Remember What You Read — Farnam Street (08/2021)

  Whenever appropriate, papers on machine learning will... be evaluated on the basis of... 1) Novelty of algorithm; 2) Novelty of application/problem; 3) Difficulty of application; 4) Quality of results; 5) Insight conveyed... Application papers should... describe work that has direct relevance to, and addresses the full complexity of, solving a non-trivial problem.

Guidelines for Writing A Good NeurIPS Paper — The NeurIPS 2006 Program Committee (2013)

  Your purpose is to communicate specific ideas, and everything about your paper should contribute to this goal... you should give away the punchline... Make your writing crisp and to the point... Avoid puffery, self-congratulation, and value judgments: give the facts and let the reader judge.

How to Write A Technical Paper — Michael D. Ernst (11/10/2018)

We’ve found that the biggest contributions at OpenAI come from cross-functional experts, so we either need to hire them or grow them here... On a typical learning day, people do things like: Reimplement papers. Follow deep learning tutorials. Play with new tools in cluster management, compilation, virtual world generation, or coding paradigms. Learn how to do research on bite-sized problems. Read about new developments in seemingly unrelated areas of AI.

How Learning Day Was Started and How It Works at OpenAI — OpenAI (08/01/2019)

  I tend... to imagine that the person who wrote this paper is very early in their career... and the last thing I want to do is to discourage that person from continuing to try... What we really want is for the scientific field to move forward... and for important new information that's going to affect the way people do things to get out there.

On Writing Quality Peer Reviews — Noah A. Smith (01/07/2019)

  Your comments should begin by summarizing the main ideas of the submission and relating these ideas to previous work... You should then summarize the strengths and weaknesses of the submission, focusing on each of the following four criteria: Quality, Clarity, Originality and Significance.

NeurIPS Reviewer, AC & SAC Guidelines — The NeurIPS Program Committee (2018)

  Our current publication system should be redesigned to maximize the rate of progress in our field. This means accelerating the speed at which new ideas and results are exchanged, disseminated, and evaluated... Ideas that turn out to be highly influential are sometimes held up for months (if not years) in reviewing purgatory, particularly if they require several years to come to maturity... The friction in our publication system is slowing the progress of our field. It makes progress incremental. And it makes our conferences somewhat boring...

A New Publishing Model in Computer Science — Yann Lecun (2011)

  Be prepared with multiple potential projects... be sure to know what the goal is, what is needed to accomplish it... Some undergraduates may prefer more direction... Your primary goal should be to ensure frequent communication... understand the approach and the implementation, at least at some level.

(Graduate Student) Collaborating with Undergraduates in Research — Michael D. Ernst (01/28/2019)

Create a process for students to request a letter and make sure they know about it well in advance... Communicate what's needed for a strong letter, what can and cannot be said... Not only comment on strengths, achievements and potential, but also try to convey why they might fit... try to comment on the individual's traits beyond the prescriptive nature of classroom or lab when you can do so meaningfully... It is important for cultivating a sense of trust with those who read the letters now and in the future... Understand how the letter may be read in the context of an entire application package via frank conversations... fill in the gaps... tell detailed stories...

Writing the Perfect Recommendation Letter -- Nature Career Column by Andy Tay (07/20/2020)

  Commit to something and commit hard. Doesn't matter if you switch later. It's easier to prove yourself if you've had to do it once before... You learn just as much, if not more, about whether you like something by actually doing it, rather than thinking about if you might like it... As years and experience in your career build up, you'll be evaluated less on your potential and more on what you've actually done.

Advice for People in Their 20s — Jean Yang (12/16/2018)

Improving your reasoning ability is probably the best investment you can make... Aim to exercise at least three times a week, maintain enough fitness so you can easily run 10k and perform standard calisthenics... Friends should push you to go further, have qualities that you are still trying to build. Stay away from negative or lazy people...

Victoria: Discern carefully between individuals who are genuinely lazy and those who face challenges that require additional time, empathy and help.

Julian Schrittwieser's Living Advices — Julian Schrittwieser

Those moments when you want to stay in bed and binge watch Netflix, those days when it feels better to blame the weather, those times when you feel like it's too much for you to handle, those are defining moments in your path towards becoming successful... It's when you're at your lowest point that you have to pick yourself up and continue, go through the discomfort, feel its pain.

Tough Days Counts More. — Anonymous

I give myself permission to thrive... I give myself permission to achieve... Now is my time to live fully... Now is my time to celebrate... Now is my time to shine... Today I affirm my abundance.

Giving Yourself Permission to Be Great. — Bob Baker