Facebook is developing a new system based on machine learning which allows users to recreate virtually any text style of any photo with just one word example, which can lead to new developments in translation, editing, reproduction, and so on.
As you can see here, Facebook TextStyleBrush project would allow users to replaces the text of existing images, including background scenes, using only one image and one word as a reference point. The process is a significant step forward in current text replication models and, as noted, may have several potential applications.
As explained Facebook:
“Unlike most artificial intelligence systems that can do this for well-defined, specialized tasks, TextStyleBrush is the first self-supervised artificial intelligence model to replace text in both handwriting and scene images – in a single sample – with a single sample word.”
In particular, Facebook notes that the project could help translate text into images, allowing users to even interpret handwritten characters in different languages (example below), while also allowing users to create personalized messages and captions alongside other applications.
But it can also be misused. This process would allow people to easily remove watermarks from photos or change parts of actual photos to change their meaning. What if, for example, someone used this to change an old university image of a political candidate who had a sign so that he was now seemingly promoting an offensive slogan instead of the actual message?
Facebook is aware of these risks, which is why it has partially released it research project, to “encourages further research and dialogue to prevent deep-fake text attacks “
“If artificial intelligence scientists and practitioners can advance opponents in building this technology, we can learn to better detect this new type of counterfeiting and build robust systems to combat it.”
Still, it looks risky. Convenient, for sure, especially if there’s a style of text you might want to recreate, and an evolving translation. But it is possible that the risks outweigh the benefits, at least in terms of wider public adoption.
Maybe then Facebook will never use it just in translation tools, but it looks like it may have added value that Facebook wants to take advantage of.
Either way, it’s an interesting project – you can read more about the project TextStyleBrush project here.