科技藝術書報討論

Topic: Real-Time User-Guided Image Colorization with Learned Deep Priors  ppt

Author: R Zhang et al.

Reference: https://dl.acm.org/doi/10.1145/3072959.3073703

Abstract: We propose a deep learning approach for user-guided image colorization. The system directly maps a grayscale image, along with sparse, local user “hints” to an output colorization with a Convolutional Neural Network (CNN). Rather than using hand-defined rules, the network propagates user edits by fusing low-level cues along with high-level semantic information, learned from large-scale data. We train on a million images, with simulated user inputs. To guide the user towards efficient input selection, the system recommends likely colors based on the input image and current user inputs. The colorization is performed in a single feed-forward pass, enabling real-time use. Even with randomly simulated user inputs, we show that the proposed system helps novice users quickly create realistic colorizations, and offers large improvements in colorization quality with just a minute of use. In addition, we demonstrate that the framework can incorporate other user “hints” to the desired colorization, showing an application to color histogram transfer.

科技藝術書報討論  ( More … )

任課教師：許素朱 教授（+其他老師）
<xn.techart@gmail.com>
課程網址 : http://www.fbilab.org/nthu/aet/seminar
上課時間 : 研究所 週(二) 18:30AM-20:30PM
上課地點：校本部 綜二603教室 (跨院碩辦公室旁)

Course Description（課程目標）
本課程帶領學生①了解國際科技藝術創作與研究的最新趨勢，從該領域最重要之學術會議、期刊及藝術展覽活動，選讀藝術論文及作品探討報告。課程由授課老師指定閱讀論文或科技藝術作品清單給學生選讀，從報告與討論交流中，讓學生獲得科技藝術領域的作品創作、技術研發能力。②課程中會擇期邀多方領域之師資一同參與，給予跨領域面向的講演或討論交流。③課程也會授予論文撰寫及投稿的關鍵能力。

Course discussion (上課方式) ■
每兩週上一次，由同學選擇閱讀國際作品或論文並做報告，課堂上老師、同學一起參與討論、講評。第3, 5,  9, 13, 17 週閱讀與報告。

第1週 (0223) – 課程介紹（※注意同學報告，不能與這門課過去同學報告過的topic重複）
第3週 (0309) – ● 學生報告：近三年穿戴科技相關藝術作品或論文
第5週 (0323)  – ● 學生報告：COVID-19 相關藝術作品或論文
第7週 (0406)  – ■ 專家演講

第9週 (0420)  – ● 學生報告：近三年 Art paper in ACM Siggraph  or  ACM MM-Art paper
第11週 (0504)  – ■ 指導碩士/博士論文撰寫架構 ，論文「抓蟲」
第13週 (0518) – ● 學生報告：近三年Social Media相關藝術作品或論文
第15週 (0601)  – ■ 專家演講
第17週 (0615)  – ● 學生報告：近三年AI Tech+Art 相關藝術作品或論文

( More … )

Arthur:Oliver Gingrich (Visiting Research Fellow, National Centre for Computer Animation), Sean Soraghan (Sound Developer), Alain Renaud (Visiting Research Fellow, Centre for Digital Entertainment), Evgenia Emets (Artist), Dario Villanueva-Ablanedo (Visual Programmer) analema.group@gmail.com 

在過去的五年中，KIMA為關於聲音和視覺的藝術和研究項目研究了聲音的視覺特性。 KIMA的先前迭代著重於音素的數字表示-物理聲音模式-作為表現的媒介。當前的發展結合了神經網絡和機器學習策略，以探索參與式音樂創作中聲音的視覺表達。該項目顯示在倫敦迴旋處的360度畫布上，促使觀眾探索自己的聲音，作為智能的實時視覺表示。機器學習算法在將聲音作為視覺形式進行有意義的解釋中起著關鍵作用。身臨其境的表演使觀眾成為作品的共同創造者。  

KIMA在牛津大學和伯恩茅斯大學展出，並在國外展出。在瑞士日內瓦青年藝術中心Fondation L’Abri，集體鼓勵音樂參與藏族歌唱碗和聲音。聽眾面對不同聲音源之間出現的實時干擾模式。 KIMA的未來發展將探索聲音，音調和樂器，共鳴與不諧調，和聲與無調之間的空間。對觀眾參與的關注極大地推動了該項目的發展，並再次強調了觀眾作為活躍表演者的作用。通過將文字集成到敘事拱門中，未來的發展也將集中在顏色，數學波動方程式上，並繼續努力在我們合作的空間的體系結構中反映聲音。作為參與式聲音探索的環境，KIMA建立了一個框架，供聽眾和藝術家以幾何，燈光，色彩或形式等不同的方式體驗聲音。 

史蒂拉克(Stelarc)是生物藝術發展史上至關重要的一位創作者。
他接受過雕塑與表演訓練，因此在數十年間，他持續嘗試著建模或重新設計自己的身體。而他的展演皆是以現場演出為主，而非使用文件展的方式，將自身表演作為附載政治或文化話語的場所。這其實貫徹著他對於自身肉體”尚未完成”的認知，我們更可以說，正因為觀者所見得的盡是他已展現的面貌，每一個頃刻皆已過往，於是所有人都是這個過往事實的見證者，而且準備往”更完整”的下一刻踏過去。這也是史蒂拉克始終將自身的演出作為創作整體，而不願意以言談或著作多加詮釋的原因。這次的報告，我以側觀書寫的方法，嘗試釐清史蒂拉克自1980年至今重要作品的創作意圖，並以現存展演文本的脈絡作為對照，解讀其作品中的展製策略。

生物藝術的概念不僅是個時間的定義，更多的指生物藝術是否運用了新的媒介、技術和是否提出新的倫理問題。也與生物藝術概念的產生和價值判斷相關。－－Eduardo Kac(1997)

在這個報告裡，我使用自身過去在生物藝術領域的檢索作為鋪墊，引用中國學者張海濤界定之泛生物藝術與生物藝術時期論述，進行兩個時期代表作品的簡略介紹。同時，依照自身觀察，統合生物藝術與科技藝術領域發展日趨同步化的趨勢，將2012年至今獨立為生物藝術第三時期，劃出機器學習發明之後對該領域的重大影響。對照來看，生物藝術所探討的問題與方法簡列如下：

（1）1933-1998 泛生物藝術。議題多環繞於對社會倫理的挑戰，也就是創作者自身之於他者的關係上。

（2）1998-2012 生物藝術的成形與發展。基因改良技術發明、電機等技術物更加精細與成熟。創作者們關注技術如何影響生活，並且嘗試打破過去自身”肉體”與機器的界線。

（3）2012-至今 發現人類的侷限性與賽博格生成。人工智慧技術迎來重大突破。強人工智慧在個別工作項目上可以做得比人類更快、更好，這引起了一陣創作者對於人類的”思覺與智慧”受到機器威脅的恐慌，也使更多創作者將目光投向未來生態情勢、賽博格體研發、推測設計等項目。

1. 3D Object Manipulation in a Single Photograph using Stock 3D Models

Arthur

Natasha Kholgade,Tomas Simon,Alexei Efros,Yaser Sheikh

Abstract

Photo-editing software restricts the control of objects in a photograph to the 2D image plane. We present a method that enables users to perform the full range of 3D manipulations, including scaling, rotation, translation, and nonrigid deformations, to an object in a photograph. As 3D manipulations often reveal parts of the object that are hidden in the original photograph, our approach uses publicly available 3D models to guide the completion of the geometry and appearance of the revealed areas of the object. The completion process leverages the structure and symmetry in the stock 3D model to factor out the effects of illumination, and to complete the appearance of the object. We demonstrate our system by producing object manipulations that would be impossible in traditional 2D photo-editing programs, such as turning a car over, making a paper-crane flap its wings, or manipulating airplanes in a historical photograph to change its story.

Publication

ACM Transaction on Graghics. July 2014.

https://doi.org/10.1145/2601097.2601209

以上期刊心得 :

1.在照片上直觀執行3D操作，無需訪問原始場景或拍攝照片的時刻。使用者可以方便地創建定格動畫來克服物理限制。此研究方法也可將其用於校正公共資料庫中3D模型的幾何形狀和外觀。

2.該研究方法的基本局限性在於採樣，如果拍攝對象的影像尺寸空間很小，可能會缺少像素樣本，在這種情況下，可能會產生模糊的結果。

3.如果3D資料庫中的模型設計不正確或設計太粗糙，可能會發生問題，尤其是在物件具有複雜幾何形狀和小零件繁瑣的情況下。

4.個人發現3D重建(3D RECONSTRUCTION)的學術研究或技術開發最大的課題還是在解決造型與光影於人類視覺經驗中的可視性問題與合理呈現問題，這也是要不斷去克服與創新的電腦科學工程。

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2 . TeeVR: spatial template-based acquisition, modeling, and rendering of large-scale indoor spaces

Arthur

Nathan Doh, Hyunga Choi, Bumchul Jang, Sangmin, Hyojin Jung, Sungkil Lee.

Abstract

Conventional image-based rendering has limited applicability for large-scale spaces. In this study, we demonstrate an efficient alternative to conventional image-based rendering. Our key approach is based on a spatial template (ST), which solely includes architectural geometric primitives. The predictability of ST improves the efficiency of acquisition, storage, and rendering. Thereby, our system can be applied to the modeling and rendering of larger indoor spaces.

Publication

ACM SIGGRAPH 2019 Emerging Technologies

July 2019 Article No.25 pp1-1

https://doi.org/10.1145/3305367.3327989

以上期刊心得

1.對於快速建立大型室內空間的模型生成，確實有很大的自動化生成成果，尤其該研究可以精準測距，提供正確的空間的尺度與比例。對於建築設計、室內設計、室內照明設計能夠提供很大的幫助。當然，此研究技術開發對於運用在電腦遊戲模擬真實世界的實際場景的建立也是非常好的場景快速生成方法。

2.很可惜的是在場景中的建立是針對所謂該研究的Spatial Template 也有就是空間中的天、地、牆板、梁柱的基本元素建立，空間中的裝置、擺設等等因運算處理速度關係，於本研究中全部去除不計考量，而場景中的許多立面畫面皆以攝影鏡頭捕捉後直接用Mapping方式，故空間中的立面層次與立體感皆扁平化而失真，較為可惜。當然，這也是值得研究下去的另一個開發課題。

1. Immersify: St. Stephen’s Cathedral illuminated – Artist Presentation

Artists

ScanLAB Projects (UK), RIEGL Laser Measurement Systems (AT), Dombauhütte St. Stephan zu Wien (AT), Ars Electronica Futurelab (AT)

Abstract

The cathedral church of St. Stephen in Vienna was spatially measured with more than 20 billion laser points. At this unique level of detail, they correspond to a data volume of almost 400 gigabytes and form the basis of the translucent St. Stephen’s Cathedral. This innovative approach offers spectators an interactive journey through the translucent sacred building in stereoscopic 8K.

The novel insights through the layers of the cathedral church were created as part of the European R&D project Immersify, which is dedicated to developing the next generation of immersive media. Over three years, the project partners have created content for highly immersive environments using various techniques and developing tools for processing the produced data.

Source :

https://ars.electronica.art/keplersgardens/en/stephansdom-artist-presentation/

2. The City as a House – A speculative exploration of the possibilities of abolishing known forms of habitation

Artists

Rebecca Merlic / VENT Gallery

Abstract

Vast amounts of pictures, sounds, videos and 3D scans are organized as environments in Rebecca Merlic’s The City as a House, in the form of an interactive visual novel. A work about the experiment of a white European 30-year-old heterosexual human living in Tokyo without inhabiting a private apartment over a period of time. A speculative exploration of the possibilities of abolishing known forms of habitation.

Source :

https://ars.electronica.art/keplersgardens/en/the-city-as-a-house/

3. Austria makes Sense – Trip to the Austrian Pavilion at the World Expo in Dubai

Abstract

From 1 October 2021 to 31 March 2021, the next World Expo will take place in Dubai under the motto “Connecting Minds, Creating the Future”. Austria is participating with the extraordinary project “Austria makes Sense”. What Austria presents and produces is not only meaningful, but also appeals to all the senses – from the overwhelming nature and unparalleled musical tradition to innovative solutions and state-of-the-art, sustainable construction concepts. An arc of tension – meaningful, sensual, contemplation – which the Austrian Pavilion in Dubai skillfully builds up. With a content concept and an architectural concept that visitors of every nation can experience directly and in an uncomplicated way. Through clever design that leaves a lasting impression. As an integral part of the exhibition, the iLab is dedicated to Austria’s innovative strength. This installation provides information about the Expo and Austria’s participation and is intended to inspire visitors.

Source :

https://ars.electronica.art/keplersgardens/en/expo-2020-dubai/

4. Who Am I?

Abstract

Who Am I? is a performative art installation that traces the relationship between a robot and the human body, between programmed and spontaneous, reproducible and irreproducible. The project takes the myth of Pygmalion as a starting point and investigates the relationship between creator/creation and their perception in a world of AI. Who Am I? is based on the interaction between human and robot, by mirroring the relationship between natural and artificial as a deep meditation on human nature.

Project Credits

Concept: Adrian Damian (RO), Director and writer: Alexandru Berceanu (RO); Set designer: Adrian Damian; Motion designer: Andrei Mitișor (RO), Sebastian Comănescu (RO); Soundscape: Vlaicu Golcea (RO); Coreography: Andrea Gavriliu (RO); Costumes: Luiza Enescu (RO), Video designer: Dilmana Yordanova (BG), Răzvan Pascu (RO), Videographers: Ioana Bodale (RO), Adi Bulboacă (RO), Sorin Florea (RO), Performers: Andrea Gavriliu (RO), Mădălina Ciotea (RO), Vlad Bîrzanu (RO), Anda Saltelechi (RO), Meda Victor (RO), Eduard Trifa (RO), Video engineer: Luca Achim (RO), Lighting designer: Cristian Simon (RO), Light engineer: Andrei Delzeg (RO)

Source :

https://ars.electronica.art/keplersgardens/en/who-am-i/

挑選的以上四部2020 ARS的作品有一個共同的特色，就是類似所謂的『沉浸式藝術』。

在表演藝術多方發展的今日，許多觀眾已不再滿足於傳統的單向觀戲模式，而創作者也運用各式媒介創造出不同的戲劇效果與體驗。近年來出現的全新劇場形式 :  沉浸式劇場（immersive theatre），就強調讓觀眾主動體驗、介入表演，跨越劇場的空間限制，打破觀眾觀看與舞台表演之間的固有界線。

假如談到沉浸式電影，通常都是指「虛擬實境電影」，當觀眾戴上VR裝置、似乎完全沉浸在另一個世界，甚至還會有互動與觸覺跟嗅覺的體驗。在強調整個身體與全方位的感官體驗的面向上，像是科技藝術的空間互動沉浸體驗，最有名的如日本team Lab的作品。但虛擬實境是藉由頭套裝置，彷彿於虛擬空間行走，而team Lab作品則是在真實空間行走，但是透過佈滿整個空間的投影和精緻的互動，沉浸到另一個世界中。

現今科技藝術的呈現思維如同沉浸式劇場，不再以表演者、故事、動作、情節為主，這並不表示它們不再重要，而是以觀眾置身在舞台的中心，強調個體觀眾的感知體驗，甚至是獨一無二、不可複製的經驗。「我」的身體感知如何與空間周遭連結，成為展演設計的概念核心。

• 科技藝術書報討論_李澤華[Alien] Star Dust: Signal to Noise – guided networked meditation (https://youtu.be/IBMLA40nGr8)
• Telluric Vibrations, UCLA Botanical Gardens – Los Angeles
• Victoria Vesna with Paul Geluso, Rhiannon Catalyst, John Brumley, Ivana Dama, Clinton Van Arman
• Crosser & LaMigra (https://youtu.be/sNSUu9NXhkk)
• Emilie Trice & LAST/RESORT present Garden del Rio Grande
• Rafael Fajardo (US)
• CINETic Residencies(https://youtu.be/rnIlIFce2HU)
• UNATC Distant Art
• Kıvanç Tatar (CA/TR), João Tragtenberg (BR), Saint Machine (RO), Leonard Nicola Hein & Claudia Schmitz (US/DE)
• sound:frame & Pausanio virtual – Area for Virtual Art(https://youtu.be/z5h1yMOM_UA)
• Ars Electronica Gallery Spaces Garden
• sound:frame / Eva Fischer (AT), Marijn Bril (NL), Angie Pohl (AT)

• 360-degree Transparent Holographic Screen Display (https://dl.acm.org/doi/10.1145/3355049.3360517)
• AlteredWind Manipulating Perceived Direction of the Wind by Cross-Modal presentation of Visual, Audio and Wind Stimuli (https://dl.acm.org/doi/10.1145/3355049.3360525)
• Biofeedback Interactive VR System Using Biological Information Measurement HMD (https://dl.acm.org/doi/10.1145/3355049.3360523)
• Brobdingnagian Glass A Micro-Stereoscopic Telexistence System (https://dl.acm.org/doi/10.1145/3355049.3360520)
• 科技藝術書報討論期末報告_李澤華

報告者：鄭涵予 PPT

文獻題目：

ARS Electronica Festival 2020 − Hybrid Art 「TOPOGRAPHIE DIGITALE」、「TIMELINES」、「DIVING INTO AN ACIDIFYING OCEAN」、「ETHEREAL FLEETING 」

作者：[Audrey Briot, Martin De Bie, Alice Giordani, CédricHonnet]、[Fabian Oefner]、[Fabian Oefner, Cristina Tarquini, Laurie Frick, Pekka Niittyvirta, Timo Aho]、[Lukas Truniger, Itamar Bergfreund,Bruce Yoder]

參考資料來源：

TOPOGRAPHIE DIGITALE—

https://ars.electronica.art/keplersgardens/en/topographie-digitale/

http://datapaulette.org/work/topographie-digitale/

TIMELINES—

https://ars.electronica.art/keplersgardens/en/timelines/

DIVING INTO AN ACIDIFYING OCEAN—

https://ars.electronica.art/keplersgardens/en/acidifying-ocean/

ETHEREAL FLEETING—

https://ars.electronica.art/keplersgardens/en/ethereal-fleeting/

作者：Akira  Nakayasu

主題 : 中安翌 Nakayasu 相關作品

參考資料來源：http://nakayasu.com/

透過了解，生物與植物之間的日常活動中不起眼、不被注意的部分，去解析且再用SMA來作出再表現的運動作用，用加熱伸縮的材料特性，去作出延伸的運動方向，而靈感就是來自地表上的植物，或是海洋的海葵，透過自然力而隨著搖擺，再加上科技層面的部分，類似聲音、光等部件結合成一項完整的作品，擷取觀眾的互動來應應作品達到一種美好的關聯，而使用形狀記憶合金運動 ( SMA )顯示技術來表達逼真的動作，將生活中所沒意識到的、常態的小事情透過不同的手法在表現，就會非常地吸引人。

報告者：趙佳禾

報告主題：Ars Electronica 2020_interactive+

報告視頻連結：

https://drive.google.com/file/d/1VGVtAJYbo5_GvbKDvZsFMinRPL95oCDW/view?usp=sharing

作品簡介：

Lauren Lee McCarthy (US): SOMEONE：SOMEONE is an interactive installation that uses our smart homes as an opportunity to shed light on the tension between intimacy and privacy as well as convenience and independent action and to ask about the role of the human workforce in a future of automation.

Mario Klingemann (DE)：Appropriate Response：Appropriate Response is an interactive installation that is concerned with meaning, expectation, and relationship with AI. The visitor is presented with a traditional wooden kneeler facing a split flat display that generates random sentences.

Lynn Hershman Leeson (US): Shadow Stalker：Shadow Stalker is a “live” interactive installation that uses original algorithms, data mining, live performances, and projections in order to make visible normally private internet information.

Dasha Ilina (RU)：Center for Technological Pain：Center for Technological Pain offers, as the artist states: “multiple practices for eliminating tech-pain.” Wrapped in a diverse range of self-built objects and solutions, and a DIY approach, an archive around workshops and tutorial videos like “Yoga for Healthy Phone Use” is created, to start a critical discussion about the negative effects that technology can have on our bodies.

(Report link)PPT
文章:Exploring Potential Scenarios and Design Implications Through A Camera-like Physical Odor Capture Prototype (paper link)
作者:Qi Lu1,2,3, Wan Liang1,2, Hao Wu1,2, Hoiian Wong1,2, Haipeng Mi1,3, Yingqing Xu
摘要:
本文主要在做一些氣味研究的探討:最近一些研究人員對尋找新的嗅覺接口輸入法。物理氣味捕獲是解決此問題的一種潛在方法，並且為了為了使用戶更容易使用，文中設計了一種便攜式基於頂空技術的快速氣味捕獲原型並受到傻瓜相機(拍立得)的啟發。作者進行了為期兩週的日記研究，共有13名參與者允許自由使用原型進行氣味捕獲活動。通過日記和訪談反饋，總結了捕獲動機，收集到的氣味等因素類型和氣味重播的感官效果。並發現捕獲活動可以積極影響用戶的情緒，記憶或知覺。用戶對設備參數的偏好做紀錄以作為指導進一步的設計迭代。氣未捕獲在日常生活和其他方面有許多潛在的應用已經提出該研究產生的影響進一步的研究。

(Report link)PPT

文章:It Is Your Turn: Collaborative Ideation with a Co-Creative Robot through Sketch
(Paper Link)
作者:Yuyu Lin1, Jiahao Guo1, Yang Chen1, Cheng Yao1, Fangtian Ying2
摘要:
本文章主旨在強調未來如何與AI 共創作一個討論式的素描描繪，共創系統已在藝術應用領域得到了廣泛的探索計算創造力。 但現有的AI合作夥伴在這裏面的角色中大多數是虛擬代理。 作為素描若帶有嵌入式機器人的紙張可能更吸引人設計師的早期構想和協作實踐，
文章中預想了移動機器人Cobbie的可能性，通過產生創意和多樣的草圖。 評估共同創造力的差異和共同創造型機器人之間的用戶體驗虛擬代理，作者進行了對比實驗，分析了從量化尺度收集的數據，觀察和半結構化訪談。 結果揭示了Cobbie在激勵方面更令人滿意探索挑出意想不到的想法並參與其中設計師在協作構思過程中。 基於這些發現，文章討論了共同創造的前景機器人用於人類-AI協作的未來發展系統。