CWRU researchers developing technique that could identify fake artworks using artificial intelligence

CWRU researchers developing technique that could identify fake artworks using artificial intelligence

CLEVELAND, Ohio — Art forgers of the world, beware.

A team of art historians and scientists at Case Western Reserve University has developed a computer technique that can identify with near certainty which artist made a particular painting based on tiny details of brush marks that can’t be controlled by the artist and aren’t visible to the naked eye.

The method combines data from the precise, three-dimensional mapping of a painting’s surface with analysis through artificial intelligence — a computer system based on the human brain and nervous system that can learn to identify and compare patterns.

The CWRU team, which first reported its findings in November in the journal Heritage Science, believes its work is breaking new ground and could be used in the future to identify fakes by spotting differences in telltale marks as tiny as the width of a brush bristle.

CWRU physics professor Kenneth Singer, who is working on the project, said such traces are indicators of what he called an artist’s “unintentional style.”

“I wouldn’t say it’s foolproof; I’m a scientist. But I would say it’s a powerful tool,” said Singer, the faculty director of MORE, the Materials for Opto/Electronics Research and Education at the university.

Michael Hinczewski, a CWRU associate professor of physics, who also serves on the research team, said in a news release that the new algorithm is so precise that it’s “almost like a fingerprint.”

Elizabeth Bolman, chairman of the art history department at CWRU, said the new methodology has the potential to vastly improve the attribution of artworks.

That’s a critical point of interest in the art market, at a time when millions of dollars could hinge on expert opinions over the authenticity of a particular object.

“We’re at the point where we’ve just figured out the basics of a concept and our first attempt ended up being spectacularly successful beyond our wildest dreams,’’ Bolman said. “Where this goes from here, we can all dream.”

High accuracy

In the experiment published in Heritage Science, Singer and Bolman and their colleagues were able to identify with greater than 95 percent accuracy which of four art students at the Cleveland Institute of Art painted nearly identical paintings of a yellow flower blossom, using the same brushes, paints, and canvas.

CWRU researchers developing technique that could identify fake artworks using artificial intelligence

A graphic from an article published in November in the journal Heritage Science was part of a paper in which multidisciplinary scholars from Case Western Reserve University used artificial intelligence algorithms to correctly attribute four nearly identical flower paintings by students at the Cleveland Institute of Art. The technique combined detailed 3-D mapping of the surface of the paintings with machine-learning analysis.Courtesy Case Western Reserve University

The analysts scanned the surfaces of the paintings and divided them digitally into grids of tiny squares from a half-millimeter to a few centimeters wide. The randomized data were then examined by the machine-learning software, which drew comparisons and then identified the four artists with high accuracy.

The CWRU project isn’t the first to use artificial intelligence to analyze works of art. Researchers at Rutgers in 2017 published a study in which they gathered data on more than 80,000 individual strokes in 300 drawings by Pablo Picasso, Henri Matisse, and Egon Schiele, and other artists, and reliably identified fakes.

But the CWRU team said they believe their project is the first to combine three-dimensional surface topography of artworks with machine learning analysis.

In a new phase of work that has yet to be published, the CWRU team used the new technology to identify correctly which portions of an early 17th-century portrait of Cardinal Tavera by El Greco, owned by a historic hospital in Toledo, Spain, were restored after the painting was cut up in pieces during the Spanish Civil War.

CWRU researchers developing technique that could identify fake artworks using artificial intelligence

A computer analysis technology under development by multidisciplinary researchers at Case Western Reserve University used computer artificial intelligence to accurately identify which portions of this 17th century portrait of “Cardinal Tavera” by El Greco were damaged and repaired by conservators.Courtesy Case Western Reserve University

Next, the team wants to compare two nearly identical versions of the crucifixion of Christ by El Greco, to see which portions were painted by the artist himself, which were painted by his son, Jorge Manuel, and which were painted by members of the artist’s workshop or treated later by conservators.

One version is owned by the Cleveland Museum of Art, and the other is owned by the Institute for Spanish and Hispanic art in Bishop Auckland, England.

“The El Greco project is looking at several different scans of paintings to see if we can identify the workshop process and identify different hands,’’ Bolman said. “Did he work on them? How much did his son Jorge work on them? These are hotly contested issues.’’

CWRU researchers developing technique that could identify fake artworks using artificial intelligence

The Cleveland Museum of Art’s early 17th-century painting of “Christ on the Cross” by El Greco is part of a multidisciplinary study of how artificial intelligence can be used to attribute authorship of artworks by measuring and comparing tiny details of individual brushstrokes that can’t be controlled by the artist or seen by the naked eye.

To meet the high demand for their paintings, artists such as El Greco, Peter Paul Rubens, and Rembrandt employed large workshops, at times making multiple versions of the same image. Scholars have been embroiled for decades in extended debates over how to attribute such works, which differ from attempts by modern-day forgers to deceive buyers by selling fakes.

The early results of the CWRU project appear to raise the possibility that computers could eliminate the need for connoisseurship, a branch of art history dedicated to identifying who made what.

But Lauryn Smith, a Ph.D. candidate in art history at CWRU and a fellow in digital art history at the Frick Museum in New York, who helped design the experiment published in Heritage Science, said that using artificial intelligence is a logical next step in the history of connoisseurship, not the end of it.

Roots of sleuthing

The field was developed in the late 19th century by Giovanni Morelli, an Italian physician, and art collector, who reasoned that artworks could be identified by studying how particular artists in the Renaissance painted “unconscious” or “invariant” details, such as hands, feet, or ears.

CWRU researchers developing technique that could identify fake artworks using artificial intelligence

Before and after images depict an El Greco portrait of a 17-century Spanish Cardinal. Researchers at Case Western Reserve University have developed an artificial intelligence technique that correctly identified areas where the damaged painting was repaired by conservators.Courtesy Case Western Reserve University

Morelli and scholars he helped to train, including the American art historian Bernard Berenson, applied the methodology to sift original paintings by Italian Old Masters from works by assistants or lesser masters.

More recently, art historians trying to determine the authenticity of artworks have combined connoisseurship with scientific data based on the age or composition of pigments, canvas, wood, or other materials.

Smith said the new machine-learning techniques are taking Morelli’s concept of “invariant” details to a new, higher level of scientific specificity.

CWRU researchers developing technique that could identify fake artworks using artificial intelligence

Researchers gathered at the Cleveland Museum of Art to watch the computerized scanning of a painting by El Greco as part of a project to use artificial intelligence in the attribution of artworks. From left: Clara Pinchbeck, grad student in Art History and Art; Mike Hinczewski, Warren E. Rupp Associate Professor of Physics, CWRU; Ph.D. biologist Fang Ji; CWRU physics Professor Ken Singer; Ambrose CWRU Swasey Professor of Physics; Carlos Bayod Lucini, Project Director, Factum Arte.Courtesy Case Western Reserve University

“There is a bit of fearmongering with this process,’’ she said. “It really shows that if you have a collaboration that has scientists and art historians and curators and all these stakeholders, you can create phenomenally useful projects that can move the field forward.”

Smith said she came up with the idea for the project several years ago together with Michael McMaster, then a Ph.D. candidate in physics.

As their relationship developed into a romance, Smith and McMaster decided to submit a paper to a conference on art and science in which they proposed applying machine-learning technology to the analysis of the topography of a painting’s surface — the tiny ridges and bumps created when an artist applies paint to canvas.

Their paper wasn’t accepted for the conference, but colleagues in the art history and physics departments at CWRU were intrigued and encouraged the couple to pursue the project, which continued after Smith and McMaster married in 2019.

“It’s our lab rats to lovers story,’’ Smith said.

Cranking it up

McMaster conceived the idea of using a chromatic confocal profilometer, a widely available scanning device, to analyze the CIA students’ paintings.

The team is now cranking up its analytical capability. Bolman arranged for the nonprofit Factum Foundation for Digital Technology in Conservation, based in Madrid, Spain, to bring its proprietary Lucida 3D scanner to the Cleveland Museum of Art in November to scan the El Greco crucifixion. The technology captures data smaller than a micron, Bolman said.

A similar process will soon be underway on the El Greco in England, she said.

CWRU researchers developing technique that could identify fake artworks using artificial intelligence

Researchers at Case Western Reserve University are using artificial intelligence techniques to identify minuscule details in brushstrokes to identify how El Greco collaborated with studio assistants to paint two nearly identical versions of “Christ on the Cross,” owned by the Cleveland Museum of Art, left, and a collection in Bishop Auckland, England, right.Courtesy Cleveland Museum of Art, Case Western Reserve University

The Factum Foundation, in a statement on its website, said the implications of the research completed so far by the CWRU team “are wide-reaching and game-changing. Connoisseurs will soon have a new tool to assist with thorny attribution questions for many paintings.”

Smith said that interest has been keen when she and McMaster have presented papers at academic conferences. They’ve had inquiries over whether the technology could be applied to the study of coins, textiles, sculptures, painted musical instruments, and other objects.

“We are significantly ahead of what anybody else is doing,’’ Bolman said. “This is a completely different way to approach visual material culture.”

https://www.cleveland.com/news/2021/12/cwru-researchers-developing-technique-that-could-identify-fake-artworks-using-artificial-intelligence.html

Steve Liem

Learn More →