robocrunch
François Chollet
@fchollet
Deep learning @google. Creator of Keras. Author of 'Deep Learning with Python'. Opinions are my own.
Tweets by François Chollet
I fear that AI research is being increasingly absorbed into the second category.
Shared by
François Chollet
at
5/14/2022
In general, a lot of the notation conventions that were evolved for pen & paper are not optimal in a digital-first world. MIDI vs traditional music notation comes to mind...
Shared by
François Chollet
at
5/13/2022
The worst-performing case also determines deployability for most high-value systems.
Shared by
François Chollet
at
5/12/2022
I wrote a ML-from-scratch book for people who know some Python: https://t.co/Pret3WlDnq Same concepts. No math equations. Just code and figures.
Shared by
François Chollet
at
5/12/2022
Motivation management, not the information content of your materials, is the central problem in education.
Shared by
François Chollet
at
5/12/2022
Making sure your students understand the language you're using is doubly important. Of course it's important because you want your explanations to land. But it's also important because any significant source of friction in the learning process kills the student's motivation.
Shared by
François Chollet
at
5/12/2022
I like Kaggle competitions because they're like a microworld for observing how scientific & engineering progress moves forward.
Shared by
François Chollet
at
12/8/2021
If your proposed method gets widely adopted in the research community and in industry, that tells you that 1) it works, 2) it's important. You don't need a panel of three random grad students to tell you this.
Shared by
François Chollet
at
12/4/2021
I'm honored and humbled to receive the Global Swiss AI Award! Many thanks to the jury and organizers. It's encouraging to see work on general AI evaluation get recognition :) Some thoughts I shared with Swiss magazine Netzwoche: https://t.co/vK5q5aF9s7
Shared by
François Chollet
at
12/3/2021
In reality, intelligence is open-ended and embodied, embedded in an environment, in an ecosystem. In the case of humans, it's also embedded in a culture, and externalized as social and technological systems. Cognition cannot be understood in a fragmented manner
Shared by
François Chollet
at
11/28/2021
On this Thanksgiving, I am grateful for all the people in my life who bring to it meaning and positive vibes -- my family, friends, awesome teammates, and the Keras community
Shared by
François Chollet
at
11/25/2021
If you're interested in Keras codebase internals: Luke from the Keras team posted a detailed walkthrough of the Model class. Implement a simplified version of the class yourself to understand how it all works under the hood! Check it out: https://t.co/ZhiYntA2J7
Shared by
François Chollet
at
11/24/2021
"Much like in biological systems, if you take any complicated experimental deep-learning setup, chances are you can remove a few modules (or replace some trained features with random ones) with no loss of performance."
Shared by
François Chollet
at
11/24/2021
By then, your validation performance has informed 19 choices that went into your model, and so that model is already the result of a search process (a training process, in fact) -- one specific instance among hundreds of possibilities.
Shared by
François Chollet
at
11/24/2021
Most people think of a validation set as a "weak test set": basically an evaluation set, but a bit less reliable than the final test set. It's more accurately a "weak training set": data that you use to improve your model (thus, on which your model will perform artificially well)
Shared by
François Chollet
at
11/24/2021
An exciting new machine learning competition with real-world impact!
Shared by
François Chollet
at
11/22/2021
What social media does to your ability to think deeply and gain perspective is similar to what a strobe light does to your ability to perceive space and time. A series of disconnected flashes of information, lasting half a second each, adding up to nothing.
Shared by
François Chollet
at
11/22/2021
Always try the simplest approach first, so you have a reference point to justify any increase in complexity. In ML, this typically means using a non-ML baseline as your first model.
Shared by
François Chollet
at
11/18/2021
The defining technological development of the 1995-2005 period was the rise of the consumer internet. 2005-2015: ubiquitous smartphones. 2015-2025? I would say it's the rapid development and deployment of multiple Covid vaccines. Incredible achievement.
Shared by
François Chollet
at
11/13/2021
Machine learning is the science of figuring out how to organize information. Information cartography, if you will
Shared by
François Chollet
at
11/7/2021
If you're using deep learning for an actual application (*not* a research paper, a school project, or a hobby project), are you doing:
Shared by
François Chollet
at
11/5/2021
There have been 3 successful iterations of the web so far, and many attempts at creating the next one... Web 1: static pages you access via desktop browsers Web 2: programmable pages you access via desktop browsers Web 3: programmable apps you access via mobile devices
Shared by
François Chollet
at
10/30/2021
FB's algorithmic mind-steering has only one stable attractor: extreme radicalization.
Shared by
François Chollet
at
10/22/2021
In conclusion: the only things you'll find in a DL model is what you put into it: the priors encoded in its architecture and the data it was trained on. DL models are not magic. They're big curves that fit their training samples, with some constraints on their structure.
Shared by
François Chollet
at
10/19/2021
The practical implication is that the best way to improve a deep learning model is to get more data or better data (overly noisy / inaccurate data will hurt generalization). A denser coverage of the latent manifold leads a model that generalizes better.
Shared by
François Chollet
at
10/19/2021
The fact that this property (latent space = very small subspace + continuous & structured) applies to so many problems is called the *manifold hypothesis*. This concept is central to understanding the nature of generalization in ML.
Shared by
François Chollet
at
10/19/2021
Let's dive deeper. But first: if you want to understand these ideas in-depth in a better format than a Twitter thread, grab your copy of Deep Learning with Python, 2nd edition, and read chapter 5 ("fundamentals of ML"). It covers all of this in detail. https://t.co/1luyISfgBY
Shared by
François Chollet
at
10/19/2021
Some people think it always refers to linear interpolation -- for instance, interpolating images in pixel space. In reality, it means interpolating *on a learned approximation of the latent manifold of the data*. That's a very different beast!
Shared by
François Chollet
at
10/19/2021
AI isn't just a set of algorithms & software artifacts. AI is also a field of engineering & research, with beliefs, dogma, goals, and accountability mechanisms (or lack thereof). It's a system made of people, a system that makes systems. *That's* the system that needs safeguards.
Shared by
François Chollet
at
10/19/2021
New code walkthrough on https://t.co/m6mT8SrKDD: a reproduction of the ConvMixer architecture from the "Patches are all you need" paper. Implemented by @RisingSayak https://t.co/smjGNxMv9N
Shared by
François Chollet
at
10/18/2021
The problem is very much information hygiene and the impact of our information infrastructure design choices. And because the nature of the problem is systemic, you will see the same type of issue arise again and again in many different contexts, long after Covid is finally over.
Shared by
François Chollet
at
10/17/2021
New tutorial on https://t.co/m6mT8SrKDD: learn how to evaluate arbitrary Python metrics while training your model (such as sklearn metrics), and display the results in TensorBoard. https://t.co/9S6pOpY0sG
Shared by
François Chollet
at
10/15/2021
You get a lot of mileage out of *not* consuming information that is not (or no longer) actionable for you. Information isn't free: you pay for it with pieces of your focus ability.
Shared by
François Chollet
at
10/14/2021
The results of the global survey of professional ML developers and data scientists (run by Kaggle) just dropped! Many interesting insights, including fresh data about ML framework adoption. https://t.co/1bJW1bJxrG
Shared by
François Chollet
at
10/14/2021
New tutorial on https://t.co/m6mT8SrKDD: image similarity search using TensorFlow Similarity. See how TF-Similarity makes it super easy to define and train similarity models in a few lines of code! https://t.co/SJtdDZrdRV
Shared by
François Chollet
at
10/12/2021
New tutorial on https://t.co/m6mT8SrKDD: text generation with FNet, a highly efficient BERT variant that leverages a Fourier transform-based linear mixer. https://t.co/QHZ1lAEcVr
Shared by
François Chollet
at
10/11/2021
Information hygiene is truly the issue of our time. If we don't fix it, our ability to address other major issues -- from the climate crisis to deadly pandemics -- takes a massive hit.
Shared by
François Chollet
at
10/8/2021
The conversation needs to move from "social media bad" (no) or "you're just against free speech" (no) to "the staggering lacks of ethics of FB's leadership have led to explicit & implicit technical & product choices (+ lacks of safeguards) that have created a toxic system"
Shared by
François Chollet
at
10/5/2021
Keras tip: if you're training models in unreliable environments like spot instances (or for long stretches of time), make sure to use the BackupAndRestore callback to be able to resume training where you left off, in just a couple of lines of configuration https://t.co/yapRjgaskV
Shared by
François Chollet
at
10/4/2021
Twitter is a bit like a neural shell. I type in words, press enter, and these words turn into thoughts across a distributed cluster of remote brains. Like this very thought in your brain. See how I did that?
Shared by
François Chollet
at
10/2/2021
I really believe that if you changed the outcome of a few purely random events throughout your life, you'd be a very different person now. That's a good reason not to be too attached to the specifics of who you are & pay more attention to big-picture invariants (values & purpose)
Shared by
François Chollet
at
10/2/2021
A portfolio project for undergrads? There's a 300 lines of code https://t.co/m6mT8SrKDD example that does exactly this (with better performance). Anyone with some Python experience could read it and modify it. Machine learning just keeps getting easier and more accessible.
Shared by
François Chollet
at
9/30/2021
Only multi-step reasoning (which is slow and resource-intensive) enables zero-shot adaptation to novel situations. Meanwhile, pattern recognition makes navigating well-known situations faster and more efficient.
Shared by
François Chollet
at
9/30/2021
Here's a trivial example to illustrate the difference between pattern recognition and reasoning and it impacts behavior generation: let's say you encounter, for the very first time, a glass door with ⅃⅃Uꟼ written on it.
Shared by
François Chollet
at
9/30/2021
New tutorial on https://t.co/m6mT8SrKDD: large-scale multi-label text classification -- predict the domain of an arXiv paper from its abstract. This tutorial features a brand new dataset collected from arXiv (also available on Kaggle). https://t.co/l3f2Grcw0z
Shared by
François Chollet
at
9/29/2021
New tutorial on https://t.co/m6mT8SrKDD: build an automatic speech recognition system using a CTC loss and a model inspired by DeepSpeech2. End-to-end (from raw wave files to inference with the trained model) in... *200 lines of code** 🤯https://t.co/J2OIoTBIls
Shared by
François Chollet
at
9/28/2021
I don't like the expression "working hard". First, it's not "working" in itself that matters, it's achieving outcomes. And second, work should not be hard. You should find work patterns that enable high-volume productivity without incurring a high psychological or physical cost.
Shared by
François Chollet
at
9/26/2021
Be sincere and earnest and intense, but don't take yourself too seriously. Our lives are just long sequences of random happenstances
Shared by
François Chollet
at
9/26/2021
Keras tip: to keep your Functional model construction in a consistent layer-like style, use closures. Left: with closures. Right: without.
Shared by
François Chollet
at
9/25/2021
Any problem can be solved via associative intuition (pattern recognition) if you've already seen hundreds of variants of the problem before. The value of principled reasoning lies in the ability to handle problems you've never seen before.
Shared by
François Chollet
at
9/25/2021
If you're a startup that has a software integration with Keras as part of its product, I'd love to chat to see how we can make Keras work better for your users. Please get in touch!
Shared by
François Chollet
at
9/22/2021
Identifying the symmetries and conserved quantities of a system isn't just useful for solving physics problems. It's more generally a way of thinking that enables you to make accurate predictions about a complex & uncertain situation without fully understanding what's going on
Shared by
François Chollet
at
9/22/2021
A widespread misconception about ML is that models are neutral/objective & just reflect the data fed into them. Modeling choices encode strong assumptions about the data, and further, the way you use your model in your application reflects your own understanding of the problem.
Shared by
François Chollet
at
9/20/2021
You wouldn't eat random crap handed to you by a greasy robot on the street. Being deliberate about what you eat is critical to your health. Well, being deliberate about the information you consume is critical to your mental well-being.
Shared by
François Chollet
at
9/19/2021
E.g. I disable the Twitter algorithmic timeline and the Discover feed in Android, among others. The things you *don't* use are as important as the things you do use. Information consumption should be deliberate.
Shared by
François Chollet
at
9/19/2021
New tutorial on https://t.co/m6mT8SrKDD: training a deep learning model (MIRNet) to enhance low-light images -- with much nicer results than what you'd get from a simple auto-contrast correction. https://t.co/rVGrYc2iAm
Shared by
François Chollet
at
9/17/2021
Captures a common pattern: many far-right/RETVRN folks like to exalt the Western cultural heritage, but in reality they don't appreciate it or even know it. It's just an ideological fashion statement for them. If they were into culture they'd push for funding education & the arts
Shared by
François Chollet
at
9/14/2021
The fact that the cost of housing in cities is growing faster than middle/working class income growth (due to artificial supply constraints) is one of the greatest threats to the prosperity of the middle/working class and one of the biggest drivers of inequality today.
Shared by
François Chollet
at
9/13/2021
How to spot an event that only occurs 0.18% of the time? Here's an essential technique that is used more rarely than it should: class weighting. A must for dealing with very imbalanced classification datasets, such as those you see in fraud detection. https://t.co/EahfxHfyvN
Shared by
François Chollet
at
9/11/2021
The best parts of the internet are about individual empowerment: discover, learn, connect, do, build. The worst parts are about control of the individual: time-wasting, algorithmic information diets in which consumer has no say and that controls their beliefs.
Shared by
François Chollet
at
9/11/2021
"To learn from data, you have to make assumptions about it. These assumptions define what can be learned. As such, the structure of your hypothesis space—the architecture of your model—is extremely important. It encodes [...] the prior knowledge that the model starts with" (DLwP)
Shared by
François Chollet
at
9/6/2021
A TensorFlow/Keras implementation of the additive attention Transformer paper (Wu et al) that was released a few days ago https://t.co/nxYHGlzYDS
Shared by
François Chollet
at
9/4/2021
In our example, we kept the data augmentation stage in float32 (created before setting the global policy) since it's meant to be run on CPU as part of the TF data pipeline.
Shared by
François Chollet
at
9/3/2021
Tweetorial: using mixed precision training with Keras models. All you need to do is set the dtype policy via `keras.mixed_precision.set_global_policy()`. Example: training an ImageNet model on Colab's TPUv2 (for free!) with bfloat16 mixed precision: https://t.co/Ob7nDCmqnD
Shared by
François Chollet
at
9/3/2021
Also yes, this is exactly what the data says. It's an information ecosystem issue. https://t.co/1ug02CsnAX
Shared by
François Chollet
at
8/31/2021
Twitter had a massive disinformation problem in 2016-2017 (when a lot of the content you came across originated from fleets of fake propaganda accounts), but has since been taking the problem seriously. It's doing quite well nowadays: it has rules and it enforces them
Shared by
François Chollet
at
8/29/2021
No need to invoke far-fetched psychological, cultural, or political theories. There are info ecosystems where vaccines are constantly decried as evil, while the miracle cure of the day is getting the spotlight. People who read this stuff every day believe it. How could they not?
Shared by
François Chollet
at
8/29/2021
Imagine a swarm of trillions of nanorobots that could assemble into a self-regenerating intelligent robot capable making copies of itself, exponentially... Yeah that's basically an ordinary multicellular organism
Shared by
François Chollet
at
8/26/2021
New on https://t.co/m6mT8SrKDD: building an OCR system for handwriting recognition. https://t.co/gxxNdKV3Hp Extremely concise and readable -- only 274 lines of code end-to-end.
Shared by
François Chollet
at
8/25/2021
The greatest asset of a language or framework is its ecosystem, and bundling breaking changes in a single release invalidates large swathes of your ecosystem. Always go with incremental change, so you bring the ecosystem with you. Evolution, not mass extinction.
Shared by
François Chollet
at
8/24/2021
Numbers will only help you if you're already asking the right questions and have already collected the right information. Most important decisions aren't like that.
Shared by
François Chollet
at
8/23/2021
Simplicity is the most important quality of a system for end users and for long-term maintenance. But there's usually a trade-off between simplicity and developer convenience, so unless the system has already been explicitly optimized for simplicity, it doesn't start out simple.
Shared by
François Chollet
at
8/22/2021
Unlike natural language, there's no pronunciation you have to pick up. You're not at any particular disadvantage if you start after 15.
Shared by
François Chollet
at
8/22/2021
"we can't fight disinformation because it would be censorship" is roughly on par with "we can't do anything about deadly pollution because it would interfere with freedom of entreprise" as a bad take.
Shared by
François Chollet
at
8/21/2021
And remember: the problem isn't "free speech", and never was. "Disinformation is a side effect of free speech" is a lie that the polluters want you to believe. The actual problems are the delivery mechanisms and those that exploit them. https://t.co/RvaG98h7Bs
Shared by
François Chollet
at
8/21/2021
Propaganda works, and those people are jacked into extremely well-optimized propaganda delivery systems. It's really as simple as that.
Shared by
François Chollet
at
8/21/2021
It's also not particularly insightful to point out the complete irrationality of what's happening. It has nothing to do logic and evidence, and everything to do with exposure to well-engineered disinformation. You won't fight that with logical arguments and sick burns.
Shared by
François Chollet
at
8/21/2021
The dumbest, most impotent idea of the 2010s and 2020s is the notion that you can fight disinformation with actual information. It's like fighting a forest fire by planting trees.
Shared by
François Chollet
at
8/21/2021
Minor tweaks in how the media talks about vaccines (and the mountains of evidence we have that they're safe and effective) will not solve vaccine hesitancy. It's a drop of water in an ocean of disinformation.
Shared by
François Chollet
at
8/21/2021
This would have been virtually impossible before Python 3.7. I implemented it using the TracebackType API, which is a new addition in 3.7. You can check out the code in Keras & TF
Shared by
François Chollet
at
8/17/2021
Second, we're automatically displaying context information related to Keras layer calls. What were the arguments passed? This is essential, since most errors have to do with the shape or dtype of the input tensors, or with the mask or training arguments, which are often implicit.
Shared by
François Chollet
at
8/17/2021
Let me tell you about my favorite upcoming feature of TensorFlow/Keras 2.7: a better debugging experience. We've worked on 3 things that significantly cut the time it takes to analyze and fix issues in your Keras code.
Shared by
François Chollet
at
8/17/2021
New code example on https://t.co/m6mT8SrKDD: a message-passing neural network for predicting properties of molecules given their graph. https://t.co/eYoZSmK9ib
Shared by
François Chollet
at
8/17/2021
Kaggle has just launched a new NLP competition, "Hindi and Tamil Question Answering". Let me add my own mini-contest to it: on Aug. 21 I will review the public notebooks that use Keras and I will select my favorite one. Winner get signed copy of my book. https://t.co/nnhNUFsIJI
Shared by
François Chollet
at
8/12/2021
New code walkthrough on https://t.co/m6mT8SrKDD: 3D volumetric rendering with Neural Radiance Fields (NeRF). https://t.co/oilZ2Mnjl3 Synthesize novel views of a 2D scene by learning the volumetric scene function.
Shared by
François Chollet
at
8/12/2021
The new release of TensorFlow (2.6) is now out! https://t.co/2radEdc8SZ Lots of great features in this release! Among other things, significant improvements to preprocessing layers, which are now part of the core Keras API.
Shared by
François Chollet
at
8/12/2021
Two useful Keras patterns demonstrated in this example: 1. Create custom layers by subclassing the Layer class. 2. Writing custom training loops by overriding train_step() in the Model class. Let's take a look.
Shared by
François Chollet
at
7/26/2021
Now on https://t.co/m6mT8SrKDD: tutorial on vector-quantized variational autoencoders (VQ-VAE), a type of VAE that uses a discrete latent space. Check it out! https://t.co/yhQgJYcLDp
Shared by
François Chollet
at
7/26/2021
Deep learning isn't magic, and it won't work at all on the specific problems you want it to work on, but it is capable of more than you know when it comes to problems where the manifold hypothesis applies (and that's a lot of problems)
Shared by
François Chollet
at
7/20/2021
Most people I meet overestimate what deep learning can do (it's curve-fitting, don't expect it to do discrete symbol manipulation, it will solve symbolic tasks via embedding + interpolation) and simultaneously underestimate what you can do with curve-fitting given enough data
Shared by
François Chollet
at
7/20/2021
Certain social networks willingly let themselves be used by bad actors as attack vectors on the minds of vulnerable people, with disastrous consequences (e.g. the anti-vax movement). Because it's profitable and they don't care. Not just in the US, but around the world
Shared by
François Chollet
at
7/19/2021
The human mind is highly vulnerable to certain patterns of social & psychological manipulation. Algorithm-powered social networks exploit these very patterns to maximize "engagement". In turn, bad actors exploit the algorithms to run sophisticated disinformation campaigns.
Shared by
François Chollet
at
7/19/2021
"Human nature has good & bad sides, and social media is just a reflection of human nature, so it includes the bad stuff" is not the wise take you think it is. Disinfo on social media is not an organic side-effect of free speech, it's the product of highly coordinated campaigns.
Shared by
François Chollet
at
7/19/2021
When it comes to social and ethical issues, you shouldn't think of AI as something unique and special that can be cleanly isolated from the rest of computer science and the tech industry. It's a part of the system, like databases and cheap high-resolution camera.
Shared by
François Chollet
at
7/18/2021
Before you ask, no, this is not what happens spontaneously when you let people exercise free speech. It's the result of deliberate and coordinated campaigns, often seeded by the propaganda arm of hostile nations (e.g. RT, Sputnik, etc. make a killing on French social media)
Shared by
François Chollet
at
7/17/2021
In both cases, the causal variable is exposure to far-right propaganda. The more misinformation you consume, the more radicalized you become. But the misinformation channels and their audience differ
Shared by
François Chollet
at
7/16/2021
New code walkthrough on https://t.co/m6mT8SrKDD: conditional GANs, for generating new images while controlling their appearance (e.g. by conditioning the generation process on a class). https://t.co/zoAEXfvkcW
Shared by
François Chollet
at
7/16/2021
A last advantage of the Functional API I haven't listed here is that it is much less verbose, because it is less redundant (no need to list/name each layer twice). Consider this subclassed VAE vs. an equivalent Functional model...
Shared by
François Chollet
at
7/12/2021
Many runtimes other than Python TensorFlow understand the Keras graph-of-layers format, such as TF.js, CoreML, DeepLearning4J... A high level, human-readable saving format is much easier to implement for third-party platforms.
Shared by
François Chollet
at
7/12/2021
b. Save it as a SavedModel -- which is a form of one-way export (of the TF graph) and won't let you reconstruct the exact same Python object. A graph of layers is a data structure; defining and saving it as a data structure is the intuitive thing to do.
Shared by
François Chollet
at
7/12/2021
If your model is a Python subclass, to serialize it you could either: a. Pickle the bytecode -- which it completely unsafe, won't work for production, and won't work across platforms
Shared by
François Chollet
at
7/12/2021
But there are several key advantages of the Functional approach over the subclassing approach: 1. Your model has known inputs shapes. 2. You get access to the internal connectivity graph. 3. The model is a data structure, not a piece of bytecode. Let's see what these are about.
Shared by
François Chollet
at
7/12/2021
Now, of course, you could also define such a model as a Python class. It would then look like this:
Shared by
François Chollet
at
7/12/2021
Tweetorial: the Functional API in Keras. Deep learning models are basically graphs of layers. Therefore, an intuitive API for defining deep learning models should be a *graph-definition API*. That's what the Functional API is: a Python-based DSL for graphs. It looks like this:
Shared by
François Chollet
at
7/12/2021
I've personally had pretty vile harassment campaigns run against me by people who saw my work on Keras as a threat to them. I still get occasional insult emails -- it's been going on since 2017. I'm pretty sure it started from just one person.
Shared by
François Chollet
at
7/12/2021
The best first step towards building a complex end-to-end system is to build a basic end-to-end system -- not to build a submodule of what you think the complex system should look like
Shared by
François Chollet
at
7/11/2021
The hardest problem in computer science is people; the second hardest problem is designing systems that are able to evolve & adapt to new goals & constraints over a span of many years or even decades
Shared by
François Chollet
at
7/10/2021
https://t.co/m6mT8SrKDD does 400k MAU according to GA, so in theory the actual figure may be more like 2M MAU? Which in turn would be consistent with the 2020 Stack Overflow survey statistic that 6% of professional developers use Keras (there are ~30M developers in the world)
Shared by
François Chollet
at
7/10/2021
ML libraries I've used in the past and liked (besides Keras & TensorFlow): Scikit-learn, XGBoost, and JAX. I want to learn more about JAX going forward. I haven't used it much so far
Shared by
François Chollet
at
7/10/2021
The second edition of Deep Learning with Python will be available in stores around September-October. Meanwhile, the code example notebooks are available on GitHub: https://t.co/QXQOV2YJ6g
Shared by
François Chollet
at
7/9/2021
And not conducive to building a new system that isn't just a faded Xerox of the original. (some people did try to build new search engines that way -- didn't really work out)
Shared by
François Chollet
at
7/9/2021
You will have accumulated terabytes of data about the system, and yet you will still be infinitely far away from what you could learn from a 3-paragraph explanation written by someone who actually understands the system.
Shared by
François Chollet
at
7/9/2021
You will also have gained no context about what surrounds the system -- what search engines are for, or what kind of cultural, technological and economic forces shape the company that developed this one
Shared by
François Chollet
at
7/9/2021
But you will still have no idea that these results are produced via the PageRank algorithm. In fact, you won't have learned anything useful about the system at all. You won't be able to create your own search engine. You won't even know what makes a search engine good.
Shared by
François Chollet
at
7/9/2021
Hyperparameter tuning? You're thinking small. You could be doing so much more... like architecture search. /s
Shared by
François Chollet
at
7/8/2021
New example on https://t.co/m6mT8SrKDD: timeseries classification with a Transformer model. https://t.co/xLJ5gc1W0k
Shared by
François Chollet
at
7/7/2021
Mark Zuckerberg is a deeply unethical creature. If you're still using FB/IG, you're financing the damage he's inflicting on the world, both via his disinformation-peddling, democracy-eroding, genocide-enabling newsfeed, and via his personal antics.
Shared by
François Chollet
at
7/6/2021
New code walkthrough on https://t.co/m6mT8SrKDD: Compact Convolutional Transformers, a computer-vision architecture that combines convolution and self-attention for great efficiency gains https://t.co/OwdSqhEJzL
Shared by
François Chollet
at
7/4/2021
And when AI gets good enough that it can make autonomous decisions, of course, you'll have other things to worry about.
Shared by
François Chollet
at
7/4/2021
Until AI is good enough that you'd trust it to make autonomous decisions (e.g. write your emails in your place), it should focus on pure automation. Which can't be more than one step ahead.
Shared by
François Chollet
at
7/4/2021
New code walkthrough on https://t.co/m6mT8SrKDD: Reinforcement Learning with Proximal Policy Optimization https://t.co/m3jDAZh6zc
Shared by
François Chollet
at
7/3/2021
When people ask me, "why use Keras?" (vs. TF w/o Keras, or another framework) I just say: why don't you take a few models you care about and compare side by side implementations? Try that with StyleGAN, or a VAE, or image captioning, or... literally anything.
Shared by
François Chollet
at
7/2/2021
This example is adapted from Soon-Yau Cheong's excellent book "Hands-On Image Generation with TensorFlow": https://t.co/jcsv2izLo9 -- check it out!
Shared by
François Chollet
at
7/2/2021
New example on https://t.co/m6mT8SrKDD: train a W-GAN to learn a latent space of molecule graphs, and use it to generate new molecules. https://t.co/fpDMXj081M
Shared by
François Chollet
at
7/1/2021
Manifold interpolation is like asking: what's the mid-point between a fork and a spoon (it terms of their function)? Which you can approximate if you've got a database of utensils related by pairwise functional distance
Shared by
François Chollet
at
6/29/2021
Usual reminder: when I've been saying for the past 5+ years that deep learning is interpolative, I don't mean it does linear interpolation in the original encoding space (which would be useless). It does interpolation on a low-dimensional manifold embedded in the encoding space.
Shared by
François Chollet
at
6/29/2021
I really think that any agent that is self-directed (that can set its own goals, that has intent) must feature emotions (if only inwards ones). Emotions are the expression of the gap between one's situation and one's intent.
Shared by
François Chollet
at
6/26/2021
New code walkthrough on https://t.co/m6mT8SrKDD: named entity recognition (NER) with Transformer (trained on the CoNLL 2003 dataset). Check it out: https://t.co/LGcg96Gtc3
Shared by
François Chollet
at
6/25/2021
Logically it can only be BatchNorm (only BN and dropout-like layers behave differently based on `training` value, and this model has no dropout). During training your model uses the current batch statistics and during inference it uses the moving averages learned during training.
Shared by
François Chollet
at
6/23/2021
New code walkthrough on https://t.co/m6mT8SrKDD: gradient centralization, a simple trick that can markedly speed up model convergence. Implemented in Keras in literally 10 lines of code. Don't miss it: https://t.co/4Gz2GamFmW
Shared by
François Chollet
at
6/18/2021
If you want to make a change in Keras, please open a PR in https://t.co/eu4PExJGJ8, rather than the TensorFlow repo.
Shared by
François Chollet
at
6/16/2021
New https://t.co/m6mT8SrKDD code walkthrough: classifying videos with a Transformer model. https://t.co/EljvwAvxLI
Shared by
François Chollet
at
6/15/2021
Deep learning paper titles are truly getting out of hand
Shared by
François Chollet
at
6/14/2021
CutMix is an image augmentation technique where you combine patches of different images, while modulating the labels to reflect the composition of the augmented images. See how to implement it in this new https://t.co/m6mT8SrKDD walkthrough: https://t.co/vRU9a4gLf8
Shared by
François Chollet
at
6/14/2021
In this https://t.co/m6mT8SrKDD code walkthrough, we build an image captioning Transformer -- in about 200 lines of beautiful, clean code ✨ Don't miss it: https://t.co/itUAVSAljt
Shared by
François Chollet
at
6/7/2021
Much more fun than random bug fixes and doc fixes in Keras
Shared by
François Chollet
at
6/5/2021
New code walkthrough on https://t.co/m6mT8SrKDD: video classification with a CNN-RNN architecture, trained on a dataset of short action sequences. A pretrained CNN extracts per-frame visual features, and the sequence of feature vectors is sent to a RNN. https://t.co/ZV3FqCJH0U
Shared by
François Chollet
at
6/5/2021
..."the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array" -- for use as a visual neuroprosthesis for the blind. This is incredible -- congrats Antonio!
Shared by
François Chollet
at
6/4/2021
I love hearing stories about how people / companies are using Keras (and feedback about what we can do better)! If you have a story you want to tell about what you've built, feel free to email me, any time. It's these stories that give meaning to our work
Shared by
François Chollet
at
6/4/2021
If you've purchased the advance preview of Deep Learning with Python, 2nd edition: it has just been updated with the drafts of the remaining chapters, 9 to 14.
Shared by
François Chollet
at
6/3/2021
If you want a more nuanced take on what deep learning can and cannot do, and why it definitely isn't a replacement for "programming" in any shape or form, check out this ML Street Talk episode https://t.co/dmSuqsBoo5
Shared by
François Chollet
at
6/2/2021
Usually, people who think "the mainstream media" is lying will uncritically trust information from Photoshopped pictures circulated by propaganda accounts.
Shared by
François Chollet
at
6/1/2021
A computer isn't an object, it's a medium of expression. It's semantically closer to language than to a dishwasher.
Shared by
François Chollet
at
5/31/2021
Sometimes people say, "X is technologically feasible (with deep learning), but it would require a dataset much larger than what we can collect". A more accurate take is we don't have the technology to do X given a realistically-sized dataset.
Shared by
François Chollet
at
5/31/2021
A more interesting theoretical distinction between "supervised" and "unsupervised" learning could be: 1. Forms of learning driven by an explicit, known objective vs. 2. Forms of learning driven by open-ended discovery and ever-shifting objectives elected by the system itself
Shared by
François Chollet
at
5/29/2021
Did you know you can now train TensorFlow Decision Forests models with the Keras API? https://t.co/MqSe65TETW
Shared by
François Chollet
at
5/26/2021
TensorFlow Cloud has a new documentation site on https://t.co/jZhirxlNpY: https://t.co/thgTgcXtaP Add one line to your Colab notebook / Kaggle notebook / local script and start running it on multiple GPUs/TPUs in Google Cloud. No extra configuration needed.
Shared by
François Chollet
at
5/18/2021