Altered States: 2D digital displays become 3D reality – Digital Technology Lets You Touch Great Art

Sidney Perkowitz, Candler Professor of Physics Emeritus at Emory University, writes about science and art, and other popular topics in science. His latest books are ‘Physics: A Very Short Introduction’ and ‘Real Scientists Don’t Wear Ties’.

It’s a natural impulse to reach out and touch an original artwork, perhaps to feel the strong brushstrokes in van Gogh’s Starry Night or to trace the shape of a compelling sculpture. You can’t though, and for good reason: a multitude of individual touches would soon damage the work, so museums sensibly use “Please don’t touch” signs, velvet ropes and alert guards to keep viewers at a distance. It helps that those same museums have put their collections online so you can appreciate great art right on your digital device. However, even at high resolution, images on flat screens do not clearly show surface texture or convey volumes in space. But now researchers in art and technology are finding ways for viewers to experience the texture of artworks in 2D and the solidity of those in 3D.

The missing third dimension is significant even for flat works, which typically show the texture of the background paper or canvas, or of the pigment. Some nominally two-dimensional works are inherently textured, such as Canadian artist Brian Jungen’s People’s Flag (2006), an immense vertical hanging piece made of red textiles. Helen Fielding of the University of Western Ontario has perceptively noted how vision and touch intertwine in this work:

As my eyes run across the texture of the flag, I can almost feel the textures of the materials I see; my hands know the softness of wool, the smoothness of vinyl. Though touching the work is prohibited…my hands are drawn to the fabrics, subtly reversing the priority of vision over touch…

Textural features like these are a material record of the artist’s effort that enhances a viewer’s interaction with the work. Such flat but textured works are art in “2.5D” because they extend only slightly into the third dimension. Now artworks shown in 2.5D and 3D on flat screens and as solid 3D models are giving new pleasures and insights to art lovers, curators, and scholars. As exact copies, these replicas can also help conserve fragile works while raising questions about the meaning of original art.

One approach, developed at the Swiss Federal Institute of Technology (EPFL) in Lausanne, creates a digital 2.5D image of an artwork by manipulating its lighting. Near sunset, when the sun’s rays enter a scene at an angle, small surface elevations cast long shadows that make them stand out. Similarly, the EPFL process shines a simulated light source onto a digital image. As the source is moved, it produces highlights and shadows that enhance surface details to produce a quasi-3D appearance.

This approach has links to CGI, computer-generated imagery, the technology that creates imaginary scenes and characters in science fiction and fantasy films. One powerful CGI tool is an algorithm called the bidirectional scattering distribution function (BSDF). For every point in an imagined scene, the BSDF shows how incoming light traveling in any direction would be reflected or transmitted to produce the outgoing ray seen by a viewer. The result fully describes the scene for any location of the light source.

In films, the BSDF is obtained from optical theory and the properties of the imaginary scene. The EPFL group, however, generated it from real art. In 2014, they illuminated a pane of stained glass with light from different directions and recorded the results with a high-resolution camera, creating a BSDF and showing that the method works for nearly planar items. This approach has been commercialized by Artmyn, a Swiss company co-founded by Loic Baboulaz who led the EPFL team. Artmyn makes 2.5D digital images of artworks by lighting them with LEDs at different visible wavelengths to provide color fidelity, and at infrared and ultraviolet wavelengths to further probe the surface. The result is a BSDF with up to a terabyte of data.

As an illustration, Artmyn has worked with Sotheby’s auction house to digitize two Marc Chagall works: Le Printemps (1975, oil on canvas), a village scene with a couple embracing, and Dans L’Atelier (1980, tempera on board), an artist’s studio. The Artmyn software lets a viewer zoom from the full artwork down to the fine scale of the weave of the canvas, while moving the lighting to display blobs, islands and layers of pigment. This reveals how Chagall achieves his effects and clearly illustrates the difference between oils and tempera as artistic media. Currently in process for similar digitization, Baboulaz told me, are a Leonardo da Vinci painting and a drawing, in recognition of the 500th anniversary of his death this year.

Artmyn has also digitized cultural artifacts such as a Sumerian clay tablet circa 2,000 BCE covered in cuneiform script; signatures and letters from important figures in the American Revolution; and a digital milestone, the original Apple-1 computer motherboard. These 2.5D images display signs of wear and of their creator’s presence that hugely enhance a viewer’s visceral appreciation of the real objects and their history.

For the next step, creating full 3D representations and physical replicas, the necessary data must be obtained without touching the original. One approach is LIDAR (light detection and ranging), where a laser beam is scanned over the object and reflected back to a sensor. The distance from the laser to each point on the object’s surface is found from the speed of light and its travel time, giving a map of the surface topography. LIDAR is most suitable for big artifacts such as a building façade at a coarse resolution of millimeters. Other approaches yield finer detail. In the triangulation method, for instance, a laser puts a dot of light on the object while a nearby camera records the dot’s location, giving data accurate to within 100 micrometers (0.1 millimeter). Copyists typically combine scanning methods to obtain the best surface replication and color rendition.

One big 3D copying effort is underway at the Smithsonian Institution, whose 19 museums preserve 155 million cultural and historic artifacts and artworks. Since 2013, the Smithsonian has put over 100 of these online as interactive 3D displays that can be viewed from different angles, and as data for 3D printers so people can make their own copies. The objects, chosen for popularity and diversity, include the original 1903 Wright Brothers flyer; a highly decorated 2nd century BCE Chinese incense burner; costume boots from the Broadway musical The Wiz from 1975; a mask of Abraham Lincoln’s face from shortly before his assassination in 1865; and for the 50th anniversary of the Apollo 11 moon landing, astronaut Neil Armstrong’s spacesuit. Recently added is a small 3D version of a full-sized dinosaur skeleton display at the National Museum of Natural History showing a T-rex attacking a triceratops, for which hundreds of bones were scanned by LIDAR and other methods.

A different goal animates the 3D art and technology studio Factum Arte in Madrid, Spain. Founded by British artist Adam Lowe in 2001, Factum Arte protects cultural artifacts by copying them, using its own high-resolution 3D scanning, printing and fabrication techniques.

Museums already use copies to preserve sensitive artworks on paper that need long recovery times in darkness and low humidity between showings. During these rests, the museum displays instead high-quality reproductions (and informs patrons that they are doing so). In a recent interview entitled “Datareality,” Adam Lowe expressed his similar belief that an artistically valid copy can provide a meaningful viewing experience while preserving a fragile original. One of his current projects is to replicate the tombs of the pharaohs Tutankhamun (King Tut) and Seti I, and queen Nefertari, in the Egyptian Valley of the Kings. The tombs were sealed by their builders, but once opened, they are deteriorating due to the throngs of visitors. As Lowe recently explained, “by going to see something that was designed to last for eternity, but never to be visited, you’re contributing to its destruction.”

The copies, approved by Egypt’s Supreme Council of Antiquities, will give visitors alternate sites to enter and view. At a resolution of 0.1 millimeter, the copies provide exact reproductions of the intricate colored images and text adorning thousands of square meters in the tombs. The first copy, King Tut’s burial chamber, was opened to the public in 2014, and in 2018, Factum Arte displayed its copied “Hall of Beauties” from the tomb of Seti I.

Earlier, Factum Arte had copied the huge Paolo Veronese oil on canvas The Wedding Feast at Cana (1563, 6.8 meters x 9.9 meters), which shows the biblical story where Jesus changes water into wine. The original was plundered from its church in Venice by Napoleon’s troops in 1797 and now hangs in the Louvre. The full-size copy, however, commissioned by the Louvre and an Italian foundation, was hung back at the original church site in 2007.

Factum Arte’s efforts highlight the questions that arise as exact physical copies of original art become available. Museums, after all, trade in authenticity. They offer viewers the chance to stand in the presence of a work that once felt the actual hands of its creator. But if the copy is indistinguishable from the work, does that dispel what the German cultural critic Walter Benjamin calls the “aura” of the original? In his influential 1935 essay The Work of Art in the Age of Mechanical Reproduction, he asserted that a copy lacks this aura:

In even the most perfect reproduction, one thing is lacking: the here and now of the work of art – its unique existence in a particular place. It is this unique existence – and nothing else – that bears the mark of the history to which the work has been subject.

The Factum Arte reproductions show that “original vs copy” is more nuanced than Benjamin indicates. The Egyptian authorities will charge a higher fee to enter the original tombs and a lower one for the copies, giving visitors the chance to feel the experience without causing damage. Surely this helps preserve a “unique existence in a particular place” for the original work. And for the repatriated Wedding at Cana, Lowe tellingly points out that a copy can bring its own authenticity of history and place:

Many people started to question about whether the experience of seeing [the copy] in its correct setting, with the correct light, in dialogue with this building that it was painted for, is actually more authentic than the experience of seeing the original in the Louvre.

We are only beginning to grasp what it means to have near-perfect copies of artworks, far beyond what Walter Benjamin could have imagined. One lesson is that such a copy can enhance an original rather than diminish it, by preserving it, and by recovering or extending its meaning.

Copying art by technical means has often been an unpopular idea. Two centuries ago, the English artist William Blake, known for his unique personal vision, expressed his dislike of mechanical reproduction such as imposing a grid to copy an artwork square by square. Current technology can also often stand rightfully accused of replacing the human and the intuitive with the robotic and the soulless. But properly used, today’s high-tech replications show that technology can also enlarge the power and beauty of an innately human impulse, the need to make art.

Get the Full Experience
Read the rest of this article, and view all articles in full from just £10 for 3 months.

Subscribe Today

,

No comments yet.

You must be a subscriber and logged in to leave a comment. Users of a Site License are unable to comment.

Log in Now | Subscribe Today