作者：李紹勤、Nevilla Isaac、Merve Safak

編修：蔡家丘

現今為了提升觀展體驗，整合人工智慧（以下簡稱AI）成為了一種引人注目的創新方式和參與途徑。在本次的策展過程中，我們嘗試使用前目前蔚為風潮的ChatGPT-3.5作為輔助工具，試圖探討AI對於策展工作的影響。

運用ChatGPT為畫作命名

首先，我們探索的目標是尋求如何將AI有效地整合到策展工作中。這次的「秘境——張萬傳特展 Unveil paths: A Journey Through Landscape」，不同於張萬傳知名的魚系列作品，我們展出的十八幅風景畫，畫家生前都沒有予以命名。徵得張萬傳親屬黃秋菊女士的同意後，我們的策展團隊有了為其命名的寶貴機會。

因此我們主要的目標之一是利用AI為展覽中的藝術作品生成標題。這個過程涉及運用AI演算法，分析藝術作品的各種元素，包括透過其文字敘述提供的視覺組成、主題內容和背景參考。雖然我們自己看到作品畫面後也可以簡快地命名，但是透過機器學習能力，AI系統可以產生概括每幅作品本質的標題，或許能為觀眾提供一個不同於以往的新鮮視角。

在進入本次的實作之前，讓我們再重新思考一下ChatGPT是什麼？

ChatGPT是於2022年11月由OpenAI所開發的人工智慧聊天機器人程式，通過GPT的大型語言模型訓練，當使用者輸入問題時，ChatGPT可以用人類自然對話的方式，產生相應的回覆。ChatGPT可用於資訊查詢、文字寫作（劇本、企劃、翻譯等）、程式寫作等多種領域。目前ChatGPT共兩個版本GPT-3.5及GPT-4，我們這次使用的為免費版本的GPT-3.5，必須先註冊會員才能使用。

然而我們很快就面臨難題，目前ChatGPT 3.5的版本中不提供上傳圖片並分析的功能。這讓我們想到約翰・伯格在其著作《觀看的方式》提及：「觀看先於語言。孩童先會觀看和辨識，接著才會說話。」在藝術評論與藝術研究領域中我們仰賴視覺、聽覺等感官，以及歷史背景、民族文化等知識來協助我們分析一件作品，如果ChatGPT「看不到」，那它要如何分析一件作品並為他命名呢？

透過人工描述畫作內容給「看不見」的ChatGPT

我們的組員提出了另一個解決辦法：「如果ChatGPT只能讀懂文字，為何我們不『說』一幅畫給它聽，看看它會給我們什麼答案。」

在古老的部落裡，故事或歷史便是用圖像或口述的方式傳遞下去，因此我們決定用人工描述一幅畫的畫面【圖1】給ChatGPT的方式，看看它能帶給我們什麼樣的解答。以下是其中一個實例：

ChatGPT給了十個不一樣的命名【圖2】，當中有三個令我們感到十分驚艷，分別是Solitude Amidst the Waves、The Lone Sentinel、Rock of Resilience。經過反覆討論，我們認為Solitude Amidst the Waves貼切地反映畫面中浪濤圍繞孤寂的礁岩，用此命名似乎是個不錯的選擇。

運用CLIP-interrogator輔助生成適合AI命名與繪圖的提示詞

我們依然好奇於AI如何完整的分析、建構並對作品命名。在不考慮付費的情況下，我們使用了CLIP-interrogator進行圖像分析，再交由ChatGPT 3.5重新命名，並由Leonardro.AI通過提示詞繪圖。

CLIP-interrogator可以運用模型分析圖片，生成適合AI繪圖的提示詞 。欄位中可以選擇模型參數，這次我們選擇適合圖像生成模型Stable Diffusion 2用的視覺轉換模型ViT-H。不同的模型參數可以調整提示詞的品質，品質越高所需的時間越多。

將張萬傳的風景畫Solitude Amidst the Waves（暫定）輸入並經過一段時間的等待，我們可以得到以下的提示詞：「painting of a rocky beach with waves crashing against the rocks, dulac, register, christopher, ; wide shot, color study, surf, panzer, ’20, ((water color)), 1128×191 resolution, sargent and raffaelo monti, rock star, hannah dodd, facing left, 1961, airy landscape」【圖3】

我們又將提示詞輸入至ChatGPT 3.5，看看這些提示詞能夠生成什麼樣的名字。

ChatGPT給出了「Christopher Dulac《岩石水景》」（Rocky Waterscape直譯）的答案【圖4】。首先最讓我們好奇的是Christopher Dulac究竟是何方神聖？然而經過更深入的查詢，我們確認這個世界上不存在名為Christopher Dulac的藝術家，也就是說這個名字是ChatGPT杜撰出來的。撇除AI虛構畫家的部分，單就作品名稱來看，確實是個樸素、簡潔可以代表一系列描繪海邊岩石的風景畫的名稱。這也意味著對作品命名時若不需要更複雜或概念性的含意，如馬格利特顛覆般命名的《這不是一個煙斗》，那麼AI也能為藝術品命名提供另一種可能性。

透過Leonardro.AI繪製圖片

【圖5】【圖6】這是兩張用上述的提示詞由圖像生成平台Leonardro.AI透過模型Stable Diffusion 2.1繪製的圖片，不論是水彩暈開的濃淡亦或是整體帶有底片感的暖色懷舊氛圍，若放在張萬傳那幅風景畫旁組成一個系列或許並不突兀。結合上面命名《岩石水景》的案例，我們相信AI確實是有辦法分析作品，並為作品提供一個相對客觀與簡單的名字，甚至生成風格相近的圖像作品。

即便AI能做的已經超乎預期，我們也相信AI能夠提供一部分的協助，然而如何將輸出結果有一致性地整合起來同時符合策展人的理念呢？也就是說要選擇Solitude Amidst the Waves還是Rocky Waterscape好呢？這些問題還不能完全仰賴AI解決，仍然需要我們的抉擇。至於最後與AI一同策展的成果如何呢？歡迎大家來看看！

「秘徑——張萬傳特展」

日期：2024年6月6日至7月7日（週三至週日、13:00-17:00）

地點：瑪德蓮藝廊（臺北市大安區永康街63號）  

按：本文為「與A.I.一同策展？」系列文章之二，來自臺灣師範大學藝術史研究所EMI課程"Modern Taiwanese and East Asian Art History"，敬請參照首篇說明，並理解本篇部分先以英文撰稿後帶有中譯語感處。

What would it be like to curate an exhibition with A.I.?

Translation between words and images

Author: Lee Shao-Chin, Nevilla Isaac, Merve Safak

Editor: Tsai Chia-Chiu

Use ChatGPT to name the artworks

Initially, our goal was to explore how to effectively integrate AI into exhibitions. However, in this exhibition titled “Unveil Paths: A Journey Through Landscape" featuring landscapes different from Chang Wan-Chuan’s renowned fish series, the eighteen paintings exhibited were not named by the artist during his lifetime. Ms. Huang Qiujü, a relative of Chang Wan-chuan who provided the paintings for the exhibition, gave the valuable opportunity to the team from the Graduate Institute of Art History of National Normal University to name them.

Therefore, one of the main objectives this time was to use AI to generate titles for the artworks in the exhibition. This process involved utilizing AI algorithms to analyze various elements of the artworks, including visual components, thematic content, and background references provided through written descriptions. By leveraging machine learning capabilities, the AI system could generate titles that encapsulate the essence of each artwork, perhaps offering the audience a fresh perspective previously unseen.

Before diving into the practical aspects, let’s reconsider what ChatGPT is. ChatGPT is an artificial intelligence chatbot program developed by Open AI in November 2022. Trained on large language models using the GPT framework, ChatGPT can generate responses in a human-like conversational manner when users input questions. It can be used in various fields such as information retrieval, text generation (scripts, plans, translations, etc.), and even programming. Currently, ChatGPT has two versions, GPT-3.5 and GPT-4. For this project, we are using the free version of GPT-3.5, which requires membership registration for use. 

However, our enthusiasm was tempered when we encountered a challenge. The current version of ChatGPT 3.5 does not provide the functionality to upload and analyze images. As John Berger mentioned in his work “Ways of Seeing," “Seeing comes before words. The child looks and recognizes before it can speak." In the realm of art criticism and art research, we rely on senses such as vision, hearing, as well as historical and cultural knowledge to analyze a piece of artwork. If ChatGPT cannot “see," how can it analyze a piece of artwork and name it?

Describe the content of the artworks to “blind" ChatGPT

Our team members proposed another solution: ‘If ChatGPT can only understand text, why don’t we, ‘tell’ it a picture to see what answers it will give us?'"

Indeed, in ancient tribes, stories or histories were passed down through images or oral narration. Therefore, we decided to describe a picture (fig. 1) to ChatGPT in a human-made way to see what kind of answers it could provide us with. Here’s an example:

“This is a seascape with a large rock in the middle. The waves are strong, the weather is fine, and there are few clouds."

ChatGPT provided ten different names (fig. 2), among which three impressed us greatly: ‘Solitude Amidst the Waves,’ ‘The Lone Sentinel,’ and ‘Rock of Resilience.’ After a thorough discussion, we felt that ‘Solitude Amidst the Waves’ accurately reflects the solitude of the rock in the turbulent sea in the painting. Using this as the title seemed like a great choice, because it captured the essence of the artwork well.

Use CLIP-interrogator to assist in generating prompt words suitable for AI naming and drawing

We are still curious about how AI can analyze, construct, and name artworks comprehensively. Without considering paid options, we used CLIP-interrogator for image analysis, then utilized ChatGPT 3.5 to provide names, and finally, Leonardo.AI created the drawing based on prompts.

CLIP -interrogator can analyze images through models and generate prompts suitable for AI drawing. Below, the parameter clip_model_name can be selected, this time choosing ViT-H to generate prompts suitable for Stable Diffusion 2 drawing. The model can adjust the quality of prompts, with higher quality requiring more time.

After inputting Chang Wan-chuan’s landscape painting ‘Solitude Amidst the Waves’ (tentatively) and waiting for a while, we obtained the following prompts: ‘painting of a rocky beach with waves crashing against the rocks, dulac, register, christopher, ; wide shot, color study, surf, panzer, ’20, ((water color)), 1128×191 resolution, sargent and raffaelo monti, rock star, hannah dodd, facing left, 1961, airy landscape.'(fig. 3)

Next, we input these prompts into ChatGPT 3.5 to see what kind of names these prompts can generate.

ChatGPT provided the answer “Christopher Dulac’s ‘Rocky Waterscape'" (literal translation) (fig. 4). At first, our curiosity was piqued by the name Christopher Dulac. However, further research revealed that there is no artist by that name anywhere in the world. This discovery implies that the name was fabricated by ChatGPT. Looking solely at the title without considering the AI-generated painter, it indeed is a simple and straightforward name that could represent a series of landscape paintings depicting rocky shorelines. This also suggests that if the artwork’s title doesn’t need a deeper interactive or conceptual connection, such as Magritte’s “This is not a pipe," AI can indeed offer a possibility for naming artworks.

Draw pictures with Leonardro.AI

Those images(fig. 5, fig.6 ) from Leonardro.AI based on the given prompts through Stable Diffusion 2.1 truly capture a range of aesthetics, from the subtle watercolor gradients to the overall warm and nostalgic atmosphere reminiscent of old photographs. Placed alongside Mr. Chang Wan-chuan’s landscape painting, they could indeed form a cohesive series without seeming out of place. Building on the naming example of “‘Rocky Waterscape'" we believe AI can indeed analyze artworks and provide relatively objective and straightforward names. It’s amazing how technology can enhance our understanding and appreciation of art.

While AI’s capabilities have exceeded expectations, it remains challenging to consistently align multiple tasks with a curator’s concept. Which title should we choose, Solitude Amidst the Waves or Rocky Waterscape? Although AI can offer assistance, naming and implementing these tasks cohesively poses difficulties. Thus, relying solely on AI may not entirely resolve the issue.