François Chollet (@fchollet)

François Chollet @fchollet

Deep learning @google. Creator of Keras. Author of 'Deep Learning with Python'. Opinions are my own.

Tweets by François Chollet

I fear that AI research is being increasingly absorbed into the second category.

Shared by François Chollet at 5/14/2022

In general, a lot of the notation conventions that were evolved for pen & paper are not optimal in a digital-first world. MIDI vs traditional music notation comes to mind...

Shared by François Chollet at 5/13/2022

The worst-performing case also determines deployability for most high-value systems.

Shared by François Chollet at 5/12/2022

I wrote a ML-from-scratch book for people who know some Python: https://t.co/Pret3WlDnq Same concepts. No math equations. Just code and figures.

Shared by François Chollet at 5/12/2022

Motivation management, not the information content of your materials, is the central problem in education.

Shared by François Chollet at 5/12/2022

Making sure your students understand the language you're using is doubly important. Of course it's important because you want your explanations to land. But it's also important because any significant source of friction in the learning process kills the student's motivation.

Shared by François Chollet at 5/12/2022

I like Kaggle competitions because they're like a microworld for observing how scientific & engineering progress moves forward.

Shared by François Chollet at 12/8/2021

If your proposed method gets widely adopted in the research community and in industry, that tells you that 1) it works, 2) it's important. You don't need a panel of three random grad students to tell you this.

Shared by François Chollet at 12/4/2021

I'm honored and humbled to receive the Global Swiss AI Award! Many thanks to the jury and organizers. It's encouraging to see work on general AI evaluation get recognition :) Some thoughts I shared with Swiss magazine Netzwoche: https://t.co/vK5q5aF9s7

Shared by François Chollet at 12/3/2021

In reality, intelligence is open-ended and embodied, embedded in an environment, in an ecosystem. In the case of humans, it's also embedded in a culture, and externalized as social and technological systems. Cognition cannot be understood in a fragmented manner

Shared by François Chollet at 11/28/2021

On this Thanksgiving, I am grateful for all the people in my life who bring to it meaning and positive vibes -- my family, friends, awesome teammates, and the Keras community

Shared by François Chollet at 11/25/2021

If you're interested in Keras codebase internals: Luke from the Keras team posted a detailed walkthrough of the Model class. Implement a simplified version of the class yourself to understand how it all works under the hood! Check it out: https://t.co/ZhiYntA2J7

Shared by François Chollet at 11/24/2021

"Much like in biological systems, if you take any complicated experimental deep-learning setup, chances are you can remove a few modules (or replace some trained features with random ones) with no loss of performance."

Shared by François Chollet at 11/24/2021

By then, your validation performance has informed 19 choices that went into your model, and so that model is already the result of a search process (a training process, in fact) -- one specific instance among hundreds of possibilities.

Shared by François Chollet at 11/24/2021

Most people think of a validation set as a "weak test set": basically an evaluation set, but a bit less reliable than the final test set. It's more accurately a "weak training set": data that you use to improve your model (thus, on which your model will perform artificially well)

Shared by François Chollet at 11/24/2021

An exciting new machine learning competition with real-world impact!

Shared by François Chollet at 11/22/2021

What social media does to your ability to think deeply and gain perspective is similar to what a strobe light does to your ability to perceive space and time. A series of disconnected flashes of information, lasting half a second each, adding up to nothing.

Shared by François Chollet at 11/22/2021

Always try the simplest approach first, so you have a reference point to justify any increase in complexity. In ML, this typically means using a non-ML baseline as your first model.

Shared by François Chollet at 11/18/2021

The defining technological development of the 1995-2005 period was the rise of the consumer internet. 2005-2015: ubiquitous smartphones. 2015-2025? I would say it's the rapid development and deployment of multiple Covid vaccines. Incredible achievement.

Shared by François Chollet at 11/13/2021

Machine learning is the science of figuring out how to organize information. Information cartography, if you will

Shared by François Chollet at 11/7/2021

If you're using deep learning for an actual application (*not* a research paper, a school project, or a hobby project), are you doing:

Shared by François Chollet at 11/5/2021

There have been 3 successful iterations of the web so far, and many attempts at creating the next one... Web 1: static pages you access via desktop browsers Web 2: programmable pages you access via desktop browsers Web 3: programmable apps you access via mobile devices

Shared by François Chollet at 10/30/2021

FB's algorithmic mind-steering has only one stable attractor: extreme radicalization.

Shared by François Chollet at 10/22/2021

In conclusion: the only things you'll find in a DL model is what you put into it: the priors encoded in its architecture and the data it was trained on. DL models are not magic. They're big curves that fit their training samples, with some constraints on their structure.

Shared by François Chollet at 10/19/2021

The practical implication is that the best way to improve a deep learning model is to get more data or better data (overly noisy / inaccurate data will hurt generalization). A denser coverage of the latent manifold leads a model that generalizes better.

Shared by François Chollet at 10/19/2021

The fact that this property (latent space = very small subspace + continuous & structured) applies to so many problems is called the *manifold hypothesis*. This concept is central to understanding the nature of generalization in ML.

Shared by François Chollet at 10/19/2021

Let's dive deeper. But first: if you want to understand these ideas in-depth in a better format than a Twitter thread, grab your copy of Deep Learning with Python, 2nd edition, and read chapter 5 ("fundamentals of ML"). It covers all of this in detail. https://t.co/1luyISfgBY

Shared by François Chollet at 10/19/2021

Some people think it always refers to linear interpolation -- for instance, interpolating images in pixel space. In reality, it means interpolating *on a learned approximation of the latent manifold of the data*. That's a very different beast!

Shared by François Chollet at 10/19/2021

AI isn't just a set of algorithms & software artifacts. AI is also a field of engineering & research, with beliefs, dogma, goals, and accountability mechanisms (or lack thereof). It's a system made of people, a system that makes systems. *That's* the system that needs safeguards.

Shared by François Chollet at 10/19/2021

New code walkthrough on https://t.co/m6mT8SrKDD: a reproduction of the ConvMixer architecture from the "Patches are all you need" paper. Implemented by @RisingSayak https://t.co/smjGNxMv9N

Shared by François Chollet at 10/18/2021

The problem is very much information hygiene and the impact of our information infrastructure design choices. And because the nature of the problem is systemic, you will see the same type of issue arise again and again in many different contexts, long after Covid is finally over.

Shared by François Chollet at 10/17/2021

New tutorial on https://t.co/m6mT8SrKDD: learn how to evaluate arbitrary Python metrics while training your model (such as sklearn metrics), and display the results in TensorBoard. https://t.co/9S6pOpY0sG

Shared by François Chollet at 10/15/2021

You get a lot of mileage out of *not* consuming information that is not (or no longer) actionable for you. Information isn't free: you pay for it with pieces of your focus ability.

Shared by François Chollet at 10/14/2021

The results of the global survey of professional ML developers and data scientists (run by Kaggle) just dropped! Many interesting insights, including fresh data about ML framework adoption. https://t.co/1bJW1bJxrG

Shared by François Chollet at 10/14/2021

New tutorial on https://t.co/m6mT8SrKDD: image similarity search using TensorFlow Similarity. See how TF-Similarity makes it super easy to define and train similarity models in a few lines of code! https://t.co/SJtdDZrdRV

Shared by François Chollet at 10/12/2021

New tutorial on https://t.co/m6mT8SrKDD: text generation with FNet, a highly efficient BERT variant that leverages a Fourier transform-based linear mixer. https://t.co/QHZ1lAEcVr

Shared by François Chollet at 10/11/2021

Information hygiene is truly the issue of our time. If we don't fix it, our ability to address other major issues -- from the climate crisis to deadly pandemics -- takes a massive hit.

Shared by François Chollet at 10/8/2021

The conversation needs to move from "social media bad" (no) or "you're just against free speech" (no) to "the staggering lacks of ethics of FB's leadership have led to explicit & implicit technical & product choices (+ lacks of safeguards) that have created a toxic system"

Shared by François Chollet at 10/5/2021

Keras tip: if you're training models in unreliable environments like spot instances (or for long stretches of time), make sure to use the BackupAndRestore callback to be able to resume training where you left off, in just a couple of lines of configuration https://t.co/yapRjgaskV

Shared by François Chollet at 10/4/2021

Twitter is a bit like a neural shell. I type in words, press enter, and these words turn into thoughts across a distributed cluster of remote brains. Like this very thought in your brain. See how I did that?

Shared by François Chollet at 10/2/2021

I really believe that if you changed the outcome of a few purely random events throughout your life, you'd be a very different person now. That's a good reason not to be too attached to the specifics of who you are & pay more attention to big-picture invariants (values & purpose)

Shared by François Chollet at 10/2/2021

A portfolio project for undergrads? There's a 300 lines of code https://t.co/m6mT8SrKDD example that does exactly this (with better performance). Anyone with some Python experience could read it and modify it. Machine learning just keeps getting easier and more accessible.

Shared by François Chollet at 9/30/2021

Only multi-step reasoning (which is slow and resource-intensive) enables zero-shot adaptation to novel situations. Meanwhile, pattern recognition makes navigating well-known situations faster and more efficient.

Shared by François Chollet at 9/30/2021

Here's a trivial example to illustrate the difference between pattern recognition and reasoning and it impacts behavior generation: let's say you encounter, for the very first time, a glass door with ⅃⅃Uꟼ written on it.

Shared by François Chollet at 9/30/2021

New tutorial on https://t.co/m6mT8SrKDD: large-scale multi-label text classification -- predict the domain of an arXiv paper from its abstract. This tutorial features a brand new dataset collected from arXiv (also available on Kaggle). https://t.co/l3f2Grcw0z

Shared by François Chollet at 9/29/2021

New tutorial on https://t.co/m6mT8SrKDD: build an automatic speech recognition system using a CTC loss and a model inspired by DeepSpeech2. End-to-end (from raw wave files to inference with the trained model) in... *200 lines of code** 🤯https://t.co/J2OIoTBIls

Shared by François Chollet at 9/28/2021

I don't like the expression "working hard". First, it's not "working" in itself that matters, it's achieving outcomes. And second, work should not be hard. You should find work patterns that enable high-volume productivity without incurring a high psychological or physical cost.

Shared by François Chollet at 9/26/2021

Be sincere and earnest and intense, but don't take yourself too seriously. Our lives are just long sequences of random happenstances

Shared by François Chollet at 9/26/2021

Keras tip: to keep your Functional model construction in a consistent layer-like style, use closures. Left: with closures. Right: without.

Shared by François Chollet at 9/25/2021

Any problem can be solved via associative intuition (pattern recognition) if you've already seen hundreds of variants of the problem before. The value of principled reasoning lies in the ability to handle problems you've never seen before.

Shared by François Chollet at 9/25/2021

If you're a startup that has a software integration with Keras as part of its product, I'd love to chat to see how we can make Keras work better for your users. Please get in touch!

Shared by François Chollet at 9/22/2021

Identifying the symmetries and conserved quantities of a system isn't just useful for solving physics problems. It's more generally a way of thinking that enables you to make accurate predictions about a complex & uncertain situation without fully understanding what's going on

Shared by François Chollet at 9/22/2021

A widespread misconception about ML is that models are neutral/objective & just reflect the data fed into them. Modeling choices encode strong assumptions about the data, and further, the way you use your model in your application reflects your own understanding of the problem.

Shared by François Chollet at 9/20/2021

You wouldn't eat random crap handed to you by a greasy robot on the street. Being deliberate about what you eat is critical to your health. Well, being deliberate about the information you consume is critical to your mental well-being.

Shared by François Chollet at 9/19/2021

E.g. I disable the Twitter algorithmic timeline and the Discover feed in Android, among others. The things you *don't* use are as important as the things you do use. Information consumption should be deliberate.

Shared by François Chollet at 9/19/2021

New tutorial on https://t.co/m6mT8SrKDD: training a deep learning model (MIRNet) to enhance low-light images -- with much nicer results than what you'd get from a simple auto-contrast correction. https://t.co/rVGrYc2iAm

Shared by François Chollet at 9/17/2021

Captures a common pattern: many far-right/RETVRN folks like to exalt the Western cultural heritage, but in reality they don't appreciate it or even know it. It's just an ideological fashion statement for them. If they were into culture they'd push for funding education & the arts

Shared by François Chollet at 9/14/2021

The fact that the cost of housing in cities is growing faster than middle/working class income growth (due to artificial supply constraints) is one of the greatest threats to the prosperity of the middle/working class and one of the biggest drivers of inequality today.

Shared by François Chollet at 9/13/2021

How to spot an event that only occurs 0.18% of the time? Here's an essential technique that is used more rarely than it should: class weighting. A must for dealing with very imbalanced classification datasets, such as those you see in fraud detection. https://t.co/EahfxHfyvN

Shared by François Chollet at 9/11/2021

The best parts of the internet are about individual empowerment: discover, learn, connect, do, build. The worst parts are about control of the individual: time-wasting, algorithmic information diets in which consumer has no say and that controls their beliefs.

Shared by François Chollet at 9/11/2021

"To learn from data, you have to make assumptions about it. These assumptions define what can be learned. As such, the structure of your hypothesis space—the architecture of your model—is extremely important. It encodes [...] the prior knowledge that the model starts with" (DLwP)

Shared by François Chollet at 9/6/2021

A TensorFlow/Keras implementation of the additive attention Transformer paper (Wu et al) that was released a few days ago https://t.co/nxYHGlzYDS

Shared by François Chollet at 9/4/2021

In our example, we kept the data augmentation stage in float32 (created before setting the global policy) since it's meant to be run on CPU as part of the TF data pipeline.

Shared by François Chollet at 9/3/2021

Tweetorial: using mixed precision training with Keras models. All you need to do is set the dtype policy via `keras.mixed_precision.set_global_policy()`. Example: training an ImageNet model on Colab's TPUv2 (for free!) with bfloat16 mixed precision: https://t.co/Ob7nDCmqnD

Shared by François Chollet at 9/3/2021

Also yes, this is exactly what the data says. It's an information ecosystem issue. https://t.co/1ug02CsnAX

Shared by François Chollet at 8/31/2021

Twitter had a massive disinformation problem in 2016-2017 (when a lot of the content you came across originated from fleets of fake propaganda accounts), but has since been taking the problem seriously. It's doing quite well nowadays: it has rules and it enforces them

Shared by François Chollet at 8/29/2021

No need to invoke far-fetched psychological, cultural, or political theories. There are info ecosystems where vaccines are constantly decried as evil, while the miracle cure of the day is getting the spotlight. People who read this stuff every day believe it. How could they not?

Shared by François Chollet at 8/29/2021

Imagine a swarm of trillions of nanorobots that could assemble into a self-regenerating intelligent robot capable making copies of itself, exponentially... Yeah that's basically an ordinary multicellular organism

Shared by François Chollet at 8/26/2021

New on https://t.co/m6mT8SrKDD: building an OCR system for handwriting recognition. https://t.co/gxxNdKV3Hp Extremely concise and readable -- only 274 lines of code end-to-end.

Shared by François Chollet at 8/25/2021

The greatest asset of a language or framework is its ecosystem, and bundling breaking changes in a single release invalidates large swathes of your ecosystem. Always go with incremental change, so you bring the ecosystem with you. Evolution, not mass extinction.

Shared by François Chollet at 8/24/2021

Numbers will only help you if you're already asking the right questions and have already collected the right information. Most important decisions aren't like that.

Shared by François Chollet at 8/23/2021

Simplicity is the most important quality of a system for end users and for long-term maintenance. But there's usually a trade-off between simplicity and developer convenience, so unless the system has already been explicitly optimized for simplicity, it doesn't start out simple.

Shared by François Chollet at 8/22/2021

Unlike natural language, there's no pronunciation you have to pick up. You're not at any particular disadvantage if you start after 15.

Shared by François Chollet at 8/22/2021

"we can't fight disinformation because it would be censorship" is roughly on par with "we can't do anything about deadly pollution because it would interfere with freedom of entreprise" as a bad take.

Shared by François Chollet at 8/21/2021

And remember: the problem isn't "free speech", and never was. "Disinformation is a side effect of free speech" is a lie that the polluters want you to believe. The actual problems are the delivery mechanisms and those that exploit them. https://t.co/RvaG98h7Bs

Shared by François Chollet at 8/21/2021

Propaganda works, and those people are jacked into extremely well-optimized propaganda delivery systems. It's really as simple as that.

Shared by François Chollet at 8/21/2021

It's also not particularly insightful to point out the complete irrationality of what's happening. It has nothing to do logic and evidence, and everything to do with exposure to well-engineered disinformation. You won't fight that with logical arguments and sick burns.

Shared by François Chollet at 8/21/2021

The dumbest, most impotent idea of the 2010s and 2020s is the notion that you can fight disinformation with actual information. It's like fighting a forest fire by planting trees.

Shared by François Chollet at 8/21/2021

Minor tweaks in how the media talks about vaccines (and the mountains of evidence we have that they're safe and effective) will not solve vaccine hesitancy. It's a drop of water in an ocean of disinformation.

Shared by François Chollet at 8/21/2021

This would have been virtually impossible before Python 3.7. I implemented it using the TracebackType API, which is a new addition in 3.7. You can check out the code in Keras & TF

Shared by François Chollet at 8/17/2021

Second, we're automatically displaying context information related to Keras layer calls. What were the arguments passed? This is essential, since most errors have to do with the shape or dtype of the input tensors, or with the mask or training arguments, which are often implicit.

Shared by François Chollet at 8/17/2021

Let me tell you about my favorite upcoming feature of TensorFlow/Keras 2.7: a better debugging experience. We've worked on 3 things that significantly cut the time it takes to analyze and fix issues in your Keras code.

Shared by François Chollet at 8/17/2021

New code example on https://t.co/m6mT8SrKDD: a message-passing neural network for predicting properties of molecules given their graph. https://t.co/eYoZSmK9ib

Shared by François Chollet at 8/17/2021

Kaggle has just launched a new NLP competition, "Hindi and Tamil Question Answering". Let me add my own mini-contest to it: on Aug. 21 I will review the public notebooks that use Keras and I will select my favorite one. Winner get signed copy of my book. https://t.co/nnhNUFsIJI

Shared by François Chollet at 8/12/2021

New code walkthrough on https://t.co/m6mT8SrKDD: 3D volumetric rendering with Neural Radiance Fields (NeRF). https://t.co/oilZ2Mnjl3 Synthesize novel views of a 2D scene by learning the volumetric scene function.

Shared by François Chollet at 8/12/2021

The new release of TensorFlow (2.6) is now out! https://t.co/2radEdc8SZ Lots of great features in this release! Among other things, significant improvements to preprocessing layers, which are now part of the core Keras API.

Shared by François Chollet at 8/12/2021

Two useful Keras patterns demonstrated in this example: 1. Create custom layers by subclassing the Layer class. 2. Writing custom training loops by overriding train_step() in the Model class. Let's take a look.

Shared by François Chollet at 7/26/2021

Now on https://t.co/m6mT8SrKDD: tutorial on vector-quantized variational autoencoders (VQ-VAE), a type of VAE that uses a discrete latent space. Check it out! https://t.co/yhQgJYcLDp

Shared by François Chollet at 7/26/2021

Deep learning isn't magic, and it won't work at all on the specific problems you want it to work on, but it is capable of more than you know when it comes to problems where the manifold hypothesis applies (and that's a lot of problems)

Shared by François Chollet at 7/20/2021

Most people I meet overestimate what deep learning can do (it's curve-fitting, don't expect it to do discrete symbol manipulation, it will solve symbolic tasks via embedding + interpolation) and simultaneously underestimate what you can do with curve-fitting given enough data

Shared by François Chollet at 7/20/2021

Certain social networks willingly let themselves be used by bad actors as attack vectors on the minds of vulnerable people, with disastrous consequences (e.g. the anti-vax movement). Because it's profitable and they don't care. Not just in the US, but around the world

Shared by François Chollet at 7/19/2021

The human mind is highly vulnerable to certain patterns of social & psychological manipulation. Algorithm-powered social networks exploit these very patterns to maximize "engagement". In turn, bad actors exploit the algorithms to run sophisticated disinformation campaigns.

Shared by François Chollet at 7/19/2021

"Human nature has good & bad sides, and social media is just a reflection of human nature, so it includes the bad stuff" is not the wise take you think it is. Disinfo on social media is not an organic side-effect of free speech, it's the product of highly coordinated campaigns.

Shared by François Chollet at 7/19/2021

When it comes to social and ethical issues, you shouldn't think of AI as something unique and special that can be cleanly isolated from the rest of computer science and the tech industry. It's a part of the system, like databases and cheap high-resolution camera.

Shared by François Chollet at 7/18/2021

Before you ask, no, this is not what happens spontaneously when you let people exercise free speech. It's the result of deliberate and coordinated campaigns, often seeded by the propaganda arm of hostile nations (e.g. RT, Sputnik, etc. make a killing on French social media)

Shared by François Chollet at 7/17/2021

In both cases, the causal variable is exposure to far-right propaganda. The more misinformation you consume, the more radicalized you become. But the misinformation channels and their audience differ

Shared by François Chollet at 7/16/2021

New code walkthrough on https://t.co/m6mT8SrKDD: conditional GANs, for generating new images while controlling their appearance (e.g. by conditioning the generation process on a class). https://t.co/zoAEXfvkcW

Shared by François Chollet at 7/16/2021

A last advantage of the Functional API I haven't listed here is that it is much less verbose, because it is less redundant (no need to list/name each layer twice). Consider this subclassed VAE vs. an equivalent Functional model...

Shared by François Chollet at 7/12/2021

Many runtimes other than Python TensorFlow understand the Keras graph-of-layers format, such as TF.js, CoreML, DeepLearning4J... A high level, human-readable saving format is much easier to implement for third-party platforms.

Shared by François Chollet at 7/12/2021

b. Save it as a SavedModel -- which is a form of one-way export (of the TF graph) and won't let you reconstruct the exact same Python object. A graph of layers is a data structure; defining and saving it as a data structure is the intuitive thing to do.

Shared by François Chollet at 7/12/2021

If your model is a Python subclass, to serialize it you could either: a. Pickle the bytecode -- which it completely unsafe, won't work for production, and won't work across platforms

Shared by François Chollet at 7/12/2021

But there are several key advantages of the Functional approach over the subclassing approach: 1. Your model has known inputs shapes. 2. You get access to the internal connectivity graph. 3. The model is a data structure, not a piece of bytecode. Let's see what these are about.

Shared by François Chollet at 7/12/2021

Now, of course, you could also define such a model as a Python class. It would then look like this:

Shared by François Chollet at 7/12/2021

Tweetorial: the Functional API in Keras. Deep learning models are basically graphs of layers. Therefore, an intuitive API for defining deep learning models should be a *graph-definition API*. That's what the Functional API is: a Python-based DSL for graphs. It looks like this:

Shared by François Chollet at 7/12/2021

I've personally had pretty vile harassment campaigns run against me by people who saw my work on Keras as a threat to them. I still get occasional insult emails -- it's been going on since 2017. I'm pretty sure it started from just one person.

Shared by François Chollet at 7/12/2021

The best first step towards building a complex end-to-end system is to build a basic end-to-end system -- not to build a submodule of what you think the complex system should look like

Shared by François Chollet at 7/11/2021

The hardest problem in computer science is people; the second hardest problem is designing systems that are able to evolve & adapt to new goals & constraints over a span of many years or even decades

Shared by François Chollet at 7/10/2021

https://t.co/m6mT8SrKDD does 400k MAU according to GA, so in theory the actual figure may be more like 2M MAU? Which in turn would be consistent with the 2020 Stack Overflow survey statistic that 6% of professional developers use Keras (there are ~30M developers in the world)

Shared by François Chollet at 7/10/2021

ML libraries I've used in the past and liked (besides Keras & TensorFlow): Scikit-learn, XGBoost, and JAX. I want to learn more about JAX going forward. I haven't used it much so far

Shared by François Chollet at 7/10/2021

The second edition of Deep Learning with Python will be available in stores around September-October. Meanwhile, the code example notebooks are available on GitHub: https://t.co/QXQOV2YJ6g

Shared by François Chollet at 7/9/2021

And not conducive to building a new system that isn't just a faded Xerox of the original. (some people did try to build new search engines that way -- didn't really work out)

Shared by François Chollet at 7/9/2021

You will have accumulated terabytes of data about the system, and yet you will still be infinitely far away from what you could learn from a 3-paragraph explanation written by someone who actually understands the system.

Shared by François Chollet at 7/9/2021

You will also have gained no context about what surrounds the system -- what search engines are for, or what kind of cultural, technological and economic forces shape the company that developed this one

Shared by François Chollet at 7/9/2021

But you will still have no idea that these results are produced via the PageRank algorithm. In fact, you won't have learned anything useful about the system at all. You won't be able to create your own search engine. You won't even know what makes a search engine good.

Shared by François Chollet at 7/9/2021

Hyperparameter tuning? You're thinking small. You could be doing so much more... like architecture search. /s

Shared by François Chollet at 7/8/2021

New example on https://t.co/m6mT8SrKDD: timeseries classification with a Transformer model. https://t.co/xLJ5gc1W0k

Shared by François Chollet at 7/7/2021

Mark Zuckerberg is a deeply unethical creature. If you're still using FB/IG, you're financing the damage he's inflicting on the world, both via his disinformation-peddling, democracy-eroding, genocide-enabling newsfeed, and via his personal antics.

Shared by François Chollet at 7/6/2021

New code walkthrough on https://t.co/m6mT8SrKDD: Compact Convolutional Transformers, a computer-vision architecture that combines convolution and self-attention for great efficiency gains https://t.co/OwdSqhEJzL

Shared by François Chollet at 7/4/2021

And when AI gets good enough that it can make autonomous decisions, of course, you'll have other things to worry about.

Shared by François Chollet at 7/4/2021

Until AI is good enough that you'd trust it to make autonomous decisions (e.g. write your emails in your place), it should focus on pure automation. Which can't be more than one step ahead.

Shared by François Chollet at 7/4/2021

New code walkthrough on https://t.co/m6mT8SrKDD: Reinforcement Learning with Proximal Policy Optimization https://t.co/m3jDAZh6zc

Shared by François Chollet at 7/3/2021

When people ask me, "why use Keras?" (vs. TF w/o Keras, or another framework) I just say: why don't you take a few models you care about and compare side by side implementations? Try that with StyleGAN, or a VAE, or image captioning, or... literally anything.

Shared by François Chollet at 7/2/2021

This example is adapted from Soon-Yau Cheong's excellent book "Hands-On Image Generation with TensorFlow": https://t.co/jcsv2izLo9 -- check it out!

Shared by François Chollet at 7/2/2021

New example on https://t.co/m6mT8SrKDD: train a W-GAN to learn a latent space of molecule graphs, and use it to generate new molecules. https://t.co/fpDMXj081M

Shared by François Chollet at 7/1/2021

Manifold interpolation is like asking: what's the mid-point between a fork and a spoon (it terms of their function)? Which you can approximate if you've got a database of utensils related by pairwise functional distance

Shared by François Chollet at 6/29/2021

Usual reminder: when I've been saying for the past 5+ years that deep learning is interpolative, I don't mean it does linear interpolation in the original encoding space (which would be useless). It does interpolation on a low-dimensional manifold embedded in the encoding space.

Shared by François Chollet at 6/29/2021

I really think that any agent that is self-directed (that can set its own goals, that has intent) must feature emotions (if only inwards ones). Emotions are the expression of the gap between one's situation and one's intent.

Shared by François Chollet at 6/26/2021

New code walkthrough on https://t.co/m6mT8SrKDD: named entity recognition (NER) with Transformer (trained on the CoNLL 2003 dataset). Check it out: https://t.co/LGcg96Gtc3

Shared by François Chollet at 6/25/2021

Logically it can only be BatchNorm (only BN and dropout-like layers behave differently based on `training` value, and this model has no dropout). During training your model uses the current batch statistics and during inference it uses the moving averages learned during training.

Shared by François Chollet at 6/23/2021

New code walkthrough on https://t.co/m6mT8SrKDD: gradient centralization, a simple trick that can markedly speed up model convergence. Implemented in Keras in literally 10 lines of code. Don't miss it: https://t.co/4Gz2GamFmW

Shared by François Chollet at 6/18/2021

If you want to make a change in Keras, please open a PR in https://t.co/eu4PExJGJ8, rather than the TensorFlow repo.

Shared by François Chollet at 6/16/2021

New https://t.co/m6mT8SrKDD code walkthrough: classifying videos with a Transformer model. https://t.co/EljvwAvxLI

Shared by François Chollet at 6/15/2021

Deep learning paper titles are truly getting out of hand

Shared by François Chollet at 6/14/2021

CutMix is an image augmentation technique where you combine patches of different images, while modulating the labels to reflect the composition of the augmented images. See how to implement it in this new https://t.co/m6mT8SrKDD walkthrough: https://t.co/vRU9a4gLf8

Shared by François Chollet at 6/14/2021

In this https://t.co/m6mT8SrKDD code walkthrough, we build an image captioning Transformer -- in about 200 lines of beautiful, clean code ✨ Don't miss it: https://t.co/itUAVSAljt

Shared by François Chollet at 6/7/2021

Much more fun than random bug fixes and doc fixes in Keras

Shared by François Chollet at 6/5/2021

New code walkthrough on https://t.co/m6mT8SrKDD: video classification with a CNN-RNN architecture, trained on a dataset of short action sequences. A pretrained CNN extracts per-frame visual features, and the sequence of feature vectors is sent to a RNN. https://t.co/ZV3FqCJH0U

Shared by François Chollet at 6/5/2021

..."the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array" -- for use as a visual neuroprosthesis for the blind. This is incredible -- congrats Antonio!

Shared by François Chollet at 6/4/2021

I love hearing stories about how people / companies are using Keras (and feedback about what we can do better)! If you have a story you want to tell about what you've built, feel free to email me, any time. It's these stories that give meaning to our work

Shared by François Chollet at 6/4/2021

If you've purchased the advance preview of Deep Learning with Python, 2nd edition: it has just been updated with the drafts of the remaining chapters, 9 to 14.

Shared by François Chollet at 6/3/2021

If you want a more nuanced take on what deep learning can and cannot do, and why it definitely isn't a replacement for "programming" in any shape or form, check out this ML Street Talk episode https://t.co/dmSuqsBoo5

Shared by François Chollet at 6/2/2021

Usually, people who think "the mainstream media" is lying will uncritically trust information from Photoshopped pictures circulated by propaganda accounts.

Shared by François Chollet at 6/1/2021

A computer isn't an object, it's a medium of expression. It's semantically closer to language than to a dishwasher.

Shared by François Chollet at 5/31/2021

Sometimes people say, "X is technologically feasible (with deep learning), but it would require a dataset much larger than what we can collect". A more accurate take is we don't have the technology to do X given a realistically-sized dataset.

Shared by François Chollet at 5/31/2021

A more interesting theoretical distinction between "supervised" and "unsupervised" learning could be: 1. Forms of learning driven by an explicit, known objective vs. 2. Forms of learning driven by open-ended discovery and ever-shifting objectives elected by the system itself

Shared by François Chollet at 5/29/2021

Did you know you can now train TensorFlow Decision Forests models with the Keras API? https://t.co/MqSe65TETW

Shared by François Chollet at 5/26/2021

TensorFlow Cloud has a new documentation site on https://t.co/jZhirxlNpY: https://t.co/thgTgcXtaP Add one line to your Colab notebook / Kaggle notebook / local script and start running it on multiple GPUs/TPUs in Google Cloud. No extra configuration needed.

Shared by François Chollet at 5/18/2021