科技藝術書報討論

Report: Link

Topic: Interactive Example-Based Terrain Authoring with Conditional Generative Adversarial Networks

Author: Éric Guérin et al.

Reference: https://dl.acm.org/doi/10.1145/3130800.3130804

Abstract:

Authoring virtual terrains presents a challenge and there is a strong need for authoring tools able to create realistic terrains with simple user-inputs and with high user control. We propose an example-based authoring pipeline that uses a set of terrain synthesizers dedicated to specific tasks. Each terrain synthesizer is a Conditional Generative Adversarial Network trained by using real-world terrains and their sketched counterparts. The training sets are built automatically with a view that the terrain synthesizers learn the generation from features that are easy to sketch. During the authoring process, the artist first creates a rough sketch of the main terrain features, such as rivers, valleys and ridges, and the algorithm automatically synthesizes a terrain corresponding to the sketch using the learned features of the training samples. Moreover, an erosion synthesizer can also generate terrain evolution by erosion at a very low computational cost. Our framework allows for an easy terrain authoring and provides a high level of realism for a minimum sketch cost. We show various examples of terrain synthesis created by experienced as well as inexperienced users who are able to design a vast variety of complex terrains in a very short time.

報告者：田子平

報告PPT：連結

文獻題目：渲染的幽靈─沉浸式虛擬儀式劇場之想像與實踐/Render Ghost─Imagination and Practice of Immersive Virtual Ritual Theater 

文獻作者：馮涵宇/Feng Han-Yu

文獻來源：臺灣博碩士論文知識加值系統/Paper

摘要：

        《Render Ghost》為一沉浸式的虛擬儀式劇場，始於對靈魂之根源與歸宿的叩問。在科技劇烈發展的情境之下，生命意識的邊界正逐漸地擴張與位移。其中，關 於將人類靈魂上傳至虛擬網路的諸種科學幻想，正逐漸脫離想像的範疇而步入現 實。靈魂伴隨著肉身而來，是否將以訊息的形式離之而去？而脫離肉身的虛擬靈魂 其主體明證性將如何變形、又生命最終將通向何方？這些提問根植於人類抗拒死亡 的原始欲望，虛擬靈魂作為一種人類之後的生命樣態勾勒出一種新的主體情境，揭開了重構後人類倫理的序幕。 

        此創作論述旨在闡釋圍繞著《Render Ghost》所衍伸出來的相關概念與問題意識，這件作品試圖通過一種後人類式的神話框架作為召喚、調動科幻敘事和儀式劇場之間的關係網路：向外，重新錨定人類位於演化與訊息歷史中的座標，反思將物與非人生命視為客體的知識系統，以此重新理解後人類如何踰越死亡的有限性及自我存有幻象的邊界；向內，回溯劇場與儀式的根源，並進一步將科技物、模控學與 虛擬的反思納入儀式劇場的想像圖譜之中，試以新媒體媒介的深入探索撞擊官能與無意識的深層，獲取知性與理性無法企及之處，挖掘精神的甬道、回到渾沌之源初，窺探演化與超越的想像。 

        Render Ghost is an immersive virtual ritual theater which started with an inquiry of the origin and home of our spirits. In the drastic development of technology, the boundary of life consciousness has been expanding and shifting. Among which, one sci-fi story about human spirits uploaded to a virtual network aspires to walk out from the fantasy and enter into reality. The spirit comes along with our physical bodies; would it leave the world in the form of information? While how would the identity certificate of disembodied spirit transform and where will the life travel eventually? These questions are rooted in the primitive desire where humankind resist death. The virtual spirit, as a new pattern of life following the state of the human, outlines a new arrives at the reconstruction of posthuman ethics. 

        This artist statement aims to interpret concepts and research questions around the work Render Ghost which takes a mythological framework of post-humanity as a summons, adjusting the relational network between sci-fi narratives and ritual theater. Externally, the work redefines the position of humankind in the process of evolution and informatic history, rethinking the knowledge system which regards objects and non-human lives as the object. With such a concept, I re-comprehend how post-human cross the line of the limitation of death and the border of self-existing fantasy. While internally, I would like to trace the origin of theater and rituals, and further impose the rethinking of technological products, cybernetics, and virtuality into the imaginative diagram of ritual theater, trying to strike the bottom of senses and unconsciousness by the exploration of new media. It allows the audience arrive place where knowledge and rationality can’t reach, digging the spiritual tunnel and returning back to the origin, exploring the imagination of evolution and transcendence. 

關鍵字：儀式劇場、後人類、模控學、神話、虛擬實境、靈魂、演化、熵

Key words: Ritual theater, Posthumanism, cybernetics, myth, virtual reality, ghost, evolution, entropy 

Report PPT:<File>

TOPIC
“Stepping Inside the Classification Cube: An Intimate Interaction with an AI System “

Authors: Avital Meshi and Angus G. Forbes

<Publication:SIGGRAPH ’20: ACM SIGGRAPH 2020 Art GalleryAugust 2020 Pages 387–393>  https://doi.org/10.1145/3386567.3388566

ABSTRACT
The Classification Cube art installation invites participants to become familiar with a machine-learning classification system. Inside a private space within the gallery, participants’ bodies are subjected to a classification process that detects their faces and estimates their age, gender, emotion and actions. Participants are also able to see how
their own classification compares with how the installation classifies a series of animated figures. Rapidly changing results encourage participants to actively perform their behavior to the system and alter the way it “sees” them. The entanglement with the system raises awareness regarding the effectiveness of machine interpretation.

Topic: Real-Time User-Guided Image Colorization with Learned Deep Priors  ppt

Author: R Zhang et al.

Reference: https://dl.acm.org/doi/10.1145/3072959.3073703

Abstract: We propose a deep learning approach for user-guided image colorization. The system directly maps a grayscale image, along with sparse, local user “hints” to an output colorization with a Convolutional Neural Network (CNN). Rather than using hand-defined rules, the network propagates user edits by fusing low-level cues along with high-level semantic information, learned from large-scale data. We train on a million images, with simulated user inputs. To guide the user towards efficient input selection, the system recommends likely colors based on the input image and current user inputs. The colorization is performed in a single feed-forward pass, enabling real-time use. Even with randomly simulated user inputs, we show that the proposed system helps novice users quickly create realistic colorizations, and offers large improvements in colorization quality with just a minute of use. In addition, we demonstrate that the framework can incorporate other user “hints” to the desired colorization, showing an application to color histogram transfer.

科技藝術書報討論  ( More … )

任課教師：許素朱 教授（+其他老師）
<xn.techart@gmail.com>
課程網址 : http://www.fbilab.org/nthu/aet/seminar
上課時間 : 研究所 週(二) 18:30AM-20:30PM
上課地點：校本部 綜二603教室 (跨院碩辦公室旁)

Course Description（課程目標）
本課程帶領學生①了解國際科技藝術創作與研究的最新趨勢，從該領域最重要之學術會議、期刊及藝術展覽活動，選讀藝術論文及作品探討報告。課程由授課老師指定閱讀論文或科技藝術作品清單給學生選讀，從報告與討論交流中，讓學生獲得科技藝術領域的作品創作、技術研發能力。②課程中會擇期邀多方領域之師資一同參與，給予跨領域面向的講演或討論交流。③課程也會授予論文撰寫及投稿的關鍵能力。

Course discussion (上課方式) ■
每兩週上一次，由同學選擇閱讀國際作品或論文並做報告，課堂上老師、同學一起參與討論、講評。第3, 5,  9, 13, 17 週閱讀與報告。

第1週 (0223) – 課程介紹（※注意同學報告，不能與這門課過去同學報告過的topic重複）
第3週 (0309) – ● 學生報告：近三年穿戴科技相關藝術作品或論文
第5週 (0323)  – ● 學生報告：COVID-19 相關藝術作品或論文
第7週 (0406)  – ■ 專家演講

第9週 (0420)  – ● 學生報告：近三年 Art paper in ACM Siggraph  or  ACM MM-Art paper
第11週 (0504)  – ■ 指導碩士/博士論文撰寫架構 ，論文「抓蟲」
第13週 (0518) – ● 學生報告：近三年Social Media相關藝術作品或論文
第15週 (0601)  – ■ 專家演講
第17週 (0615)  – ● 學生報告：近三年AI Tech+Art 相關藝術作品或論文

( More … )

Arthur:Oliver Gingrich (Visiting Research Fellow, National Centre for Computer Animation), Sean Soraghan (Sound Developer), Alain Renaud (Visiting Research Fellow, Centre for Digital Entertainment), Evgenia Emets (Artist), Dario Villanueva-Ablanedo (Visual Programmer) analema.group@gmail.com 

在過去的五年中，KIMA為關於聲音和視覺的藝術和研究項目研究了聲音的視覺特性。 KIMA的先前迭代著重於音素的數字表示-物理聲音模式-作為表現的媒介。當前的發展結合了神經網絡和機器學習策略，以探索參與式音樂創作中聲音的視覺表達。該項目顯示在倫敦迴旋處的360度畫布上，促使觀眾探索自己的聲音，作為智能的實時視覺表示。機器學習算法在將聲音作為視覺形式進行有意義的解釋中起著關鍵作用。身臨其境的表演使觀眾成為作品的共同創造者。  

KIMA在牛津大學和伯恩茅斯大學展出，並在國外展出。在瑞士日內瓦青年藝術中心Fondation L’Abri，集體鼓勵音樂參與藏族歌唱碗和聲音。聽眾面對不同聲音源之間出現的實時干擾模式。 KIMA的未來發展將探索聲音，音調和樂器，共鳴與不諧調，和聲與無調之間的空間。對觀眾參與的關注極大地推動了該項目的發展，並再次強調了觀眾作為活躍表演者的作用。通過將文字集成到敘事拱門中，未來的發展也將集中在顏色，數學波動方程式上，並繼續努力在我們合作的空間的體系結構中反映聲音。作為參與式聲音探索的環境，KIMA建立了一個框架，供聽眾和藝術家以幾何，燈光，色彩或形式等不同的方式體驗聲音。 

史蒂拉克(Stelarc)是生物藝術發展史上至關重要的一位創作者。
他接受過雕塑與表演訓練，因此在數十年間，他持續嘗試著建模或重新設計自己的身體。而他的展演皆是以現場演出為主，而非使用文件展的方式，將自身表演作為附載政治或文化話語的場所。這其實貫徹著他對於自身肉體”尚未完成”的認知，我們更可以說，正因為觀者所見得的盡是他已展現的面貌，每一個頃刻皆已過往，於是所有人都是這個過往事實的見證者，而且準備往”更完整”的下一刻踏過去。這也是史蒂拉克始終將自身的演出作為創作整體，而不願意以言談或著作多加詮釋的原因。這次的報告，我以側觀書寫的方法，嘗試釐清史蒂拉克自1980年至今重要作品的創作意圖，並以現存展演文本的脈絡作為對照，解讀其作品中的展製策略。

生物藝術的概念不僅是個時間的定義，更多的指生物藝術是否運用了新的媒介、技術和是否提出新的倫理問題。也與生物藝術概念的產生和價值判斷相關。－－Eduardo Kac(1997)

在這個報告裡，我使用自身過去在生物藝術領域的檢索作為鋪墊，引用中國學者張海濤界定之泛生物藝術與生物藝術時期論述，進行兩個時期代表作品的簡略介紹。同時，依照自身觀察，統合生物藝術與科技藝術領域發展日趨同步化的趨勢，將2012年至今獨立為生物藝術第三時期，劃出機器學習發明之後對該領域的重大影響。對照來看，生物藝術所探討的問題與方法簡列如下：

（1）1933-1998 泛生物藝術。議題多環繞於對社會倫理的挑戰，也就是創作者自身之於他者的關係上。

（2）1998-2012 生物藝術的成形與發展。基因改良技術發明、電機等技術物更加精細與成熟。創作者們關注技術如何影響生活，並且嘗試打破過去自身”肉體”與機器的界線。

（3）2012-至今 發現人類的侷限性與賽博格生成。人工智慧技術迎來重大突破。強人工智慧在個別工作項目上可以做得比人類更快、更好，這引起了一陣創作者對於人類的”思覺與智慧”受到機器威脅的恐慌，也使更多創作者將目光投向未來生態情勢、賽博格體研發、推測設計等項目。

1. 3D Object Manipulation in a Single Photograph using Stock 3D Models

Arthur

Natasha Kholgade,Tomas Simon,Alexei Efros,Yaser Sheikh

Abstract

Photo-editing software restricts the control of objects in a photograph to the 2D image plane. We present a method that enables users to perform the full range of 3D manipulations, including scaling, rotation, translation, and nonrigid deformations, to an object in a photograph. As 3D manipulations often reveal parts of the object that are hidden in the original photograph, our approach uses publicly available 3D models to guide the completion of the geometry and appearance of the revealed areas of the object. The completion process leverages the structure and symmetry in the stock 3D model to factor out the effects of illumination, and to complete the appearance of the object. We demonstrate our system by producing object manipulations that would be impossible in traditional 2D photo-editing programs, such as turning a car over, making a paper-crane flap its wings, or manipulating airplanes in a historical photograph to change its story.

Publication

ACM Transaction on Graghics. July 2014.

https://doi.org/10.1145/2601097.2601209

以上期刊心得 :

1.在照片上直觀執行3D操作，無需訪問原始場景或拍攝照片的時刻。使用者可以方便地創建定格動畫來克服物理限制。此研究方法也可將其用於校正公共資料庫中3D模型的幾何形狀和外觀。

2.該研究方法的基本局限性在於採樣，如果拍攝對象的影像尺寸空間很小，可能會缺少像素樣本，在這種情況下，可能會產生模糊的結果。

3.如果3D資料庫中的模型設計不正確或設計太粗糙，可能會發生問題，尤其是在物件具有複雜幾何形狀和小零件繁瑣的情況下。

4.個人發現3D重建(3D RECONSTRUCTION)的學術研究或技術開發最大的課題還是在解決造型與光影於人類視覺經驗中的可視性問題與合理呈現問題，這也是要不斷去克服與創新的電腦科學工程。

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

2 . TeeVR: spatial template-based acquisition, modeling, and rendering of large-scale indoor spaces

Arthur

Nathan Doh, Hyunga Choi, Bumchul Jang, Sangmin, Hyojin Jung, Sungkil Lee.

Abstract

Conventional image-based rendering has limited applicability for large-scale spaces. In this study, we demonstrate an efficient alternative to conventional image-based rendering. Our key approach is based on a spatial template (ST), which solely includes architectural geometric primitives. The predictability of ST improves the efficiency of acquisition, storage, and rendering. Thereby, our system can be applied to the modeling and rendering of larger indoor spaces.

Publication

ACM SIGGRAPH 2019 Emerging Technologies

July 2019 Article No.25 pp1-1

https://doi.org/10.1145/3305367.3327989

以上期刊心得

1.對於快速建立大型室內空間的模型生成，確實有很大的自動化生成成果，尤其該研究可以精準測距，提供正確的空間的尺度與比例。對於建築設計、室內設計、室內照明設計能夠提供很大的幫助。當然，此研究技術開發對於運用在電腦遊戲模擬真實世界的實際場景的建立也是非常好的場景快速生成方法。

2.很可惜的是在場景中的建立是針對所謂該研究的Spatial Template 也有就是空間中的天、地、牆板、梁柱的基本元素建立，空間中的裝置、擺設等等因運算處理速度關係，於本研究中全部去除不計考量，而場景中的許多立面畫面皆以攝影鏡頭捕捉後直接用Mapping方式，故空間中的立面層次與立體感皆扁平化而失真，較為可惜。當然，這也是值得研究下去的另一個開發課題。

1. Immersify: St. Stephen’s Cathedral illuminated – Artist Presentation

Artists

ScanLAB Projects (UK), RIEGL Laser Measurement Systems (AT), Dombauhütte St. Stephan zu Wien (AT), Ars Electronica Futurelab (AT)

Abstract

The cathedral church of St. Stephen in Vienna was spatially measured with more than 20 billion laser points. At this unique level of detail, they correspond to a data volume of almost 400 gigabytes and form the basis of the translucent St. Stephen’s Cathedral. This innovative approach offers spectators an interactive journey through the translucent sacred building in stereoscopic 8K.

The novel insights through the layers of the cathedral church were created as part of the European R&D project Immersify, which is dedicated to developing the next generation of immersive media. Over three years, the project partners have created content for highly immersive environments using various techniques and developing tools for processing the produced data.

Source :

https://ars.electronica.art/keplersgardens/en/stephansdom-artist-presentation/

2. The City as a House – A speculative exploration of the possibilities of abolishing known forms of habitation

Artists

Rebecca Merlic / VENT Gallery

Abstract

Vast amounts of pictures, sounds, videos and 3D scans are organized as environments in Rebecca Merlic’s The City as a House, in the form of an interactive visual novel. A work about the experiment of a white European 30-year-old heterosexual human living in Tokyo without inhabiting a private apartment over a period of time. A speculative exploration of the possibilities of abolishing known forms of habitation.

Source :

https://ars.electronica.art/keplersgardens/en/the-city-as-a-house/

3. Austria makes Sense – Trip to the Austrian Pavilion at the World Expo in Dubai

Abstract

From 1 October 2021 to 31 March 2021, the next World Expo will take place in Dubai under the motto “Connecting Minds, Creating the Future”. Austria is participating with the extraordinary project “Austria makes Sense”. What Austria presents and produces is not only meaningful, but also appeals to all the senses – from the overwhelming nature and unparalleled musical tradition to innovative solutions and state-of-the-art, sustainable construction concepts. An arc of tension – meaningful, sensual, contemplation – which the Austrian Pavilion in Dubai skillfully builds up. With a content concept and an architectural concept that visitors of every nation can experience directly and in an uncomplicated way. Through clever design that leaves a lasting impression. As an integral part of the exhibition, the iLab is dedicated to Austria’s innovative strength. This installation provides information about the Expo and Austria’s participation and is intended to inspire visitors.

Source :

https://ars.electronica.art/keplersgardens/en/expo-2020-dubai/

4. Who Am I?

Abstract

Who Am I? is a performative art installation that traces the relationship between a robot and the human body, between programmed and spontaneous, reproducible and irreproducible. The project takes the myth of Pygmalion as a starting point and investigates the relationship between creator/creation and their perception in a world of AI. Who Am I? is based on the interaction between human and robot, by mirroring the relationship between natural and artificial as a deep meditation on human nature.

Project Credits

Concept: Adrian Damian (RO), Director and writer: Alexandru Berceanu (RO); Set designer: Adrian Damian; Motion designer: Andrei Mitișor (RO), Sebastian Comănescu (RO); Soundscape: Vlaicu Golcea (RO); Coreography: Andrea Gavriliu (RO); Costumes: Luiza Enescu (RO), Video designer: Dilmana Yordanova (BG), Răzvan Pascu (RO), Videographers: Ioana Bodale (RO), Adi Bulboacă (RO), Sorin Florea (RO), Performers: Andrea Gavriliu (RO), Mădălina Ciotea (RO), Vlad Bîrzanu (RO), Anda Saltelechi (RO), Meda Victor (RO), Eduard Trifa (RO), Video engineer: Luca Achim (RO), Lighting designer: Cristian Simon (RO), Light engineer: Andrei Delzeg (RO)

Source :

https://ars.electronica.art/keplersgardens/en/who-am-i/

挑選的以上四部2020 ARS的作品有一個共同的特色，就是類似所謂的『沉浸式藝術』。

在表演藝術多方發展的今日，許多觀眾已不再滿足於傳統的單向觀戲模式，而創作者也運用各式媒介創造出不同的戲劇效果與體驗。近年來出現的全新劇場形式 :  沉浸式劇場（immersive theatre），就強調讓觀眾主動體驗、介入表演，跨越劇場的空間限制，打破觀眾觀看與舞台表演之間的固有界線。

假如談到沉浸式電影，通常都是指「虛擬實境電影」，當觀眾戴上VR裝置、似乎完全沉浸在另一個世界，甚至還會有互動與觸覺跟嗅覺的體驗。在強調整個身體與全方位的感官體驗的面向上，像是科技藝術的空間互動沉浸體驗，最有名的如日本team Lab的作品。但虛擬實境是藉由頭套裝置，彷彿於虛擬空間行走，而team Lab作品則是在真實空間行走，但是透過佈滿整個空間的投影和精緻的互動，沉浸到另一個世界中。

現今科技藝術的呈現思維如同沉浸式劇場，不再以表演者、故事、動作、情節為主，這並不表示它們不再重要，而是以觀眾置身在舞台的中心，強調個體觀眾的感知體驗，甚至是獨一無二、不可複製的經驗。「我」的身體感知如何與空間周遭連結，成為展演設計的概念核心。

• 科技藝術書報討論_李澤華[Alien] Star Dust: Signal to Noise – guided networked meditation (https://youtu.be/IBMLA40nGr8)
• Telluric Vibrations, UCLA Botanical Gardens – Los Angeles
• Victoria Vesna with Paul Geluso, Rhiannon Catalyst, John Brumley, Ivana Dama, Clinton Van Arman
• Crosser & LaMigra (https://youtu.be/sNSUu9NXhkk)
• Emilie Trice & LAST/RESORT present Garden del Rio Grande
• Rafael Fajardo (US)
• CINETic Residencies(https://youtu.be/rnIlIFce2HU)
• UNATC Distant Art
• Kıvanç Tatar (CA/TR), João Tragtenberg (BR), Saint Machine (RO), Leonard Nicola Hein & Claudia Schmitz (US/DE)
• sound:frame & Pausanio virtual – Area for Virtual Art(https://youtu.be/z5h1yMOM_UA)
• Ars Electronica Gallery Spaces Garden
• sound:frame / Eva Fischer (AT), Marijn Bril (NL), Angie Pohl (AT)

• 360-degree Transparent Holographic Screen Display (https://dl.acm.org/doi/10.1145/3355049.3360517)
• AlteredWind Manipulating Perceived Direction of the Wind by Cross-Modal presentation of Visual, Audio and Wind Stimuli (https://dl.acm.org/doi/10.1145/3355049.3360525)
• Biofeedback Interactive VR System Using Biological Information Measurement HMD (https://dl.acm.org/doi/10.1145/3355049.3360523)
• Brobdingnagian Glass A Micro-Stereoscopic Telexistence System (https://dl.acm.org/doi/10.1145/3355049.3360520)
• 科技藝術書報討論期末報告_李澤華

報告者：鄭涵予 PPT

文獻題目：

ARS Electronica Festival 2020 − Hybrid Art 「TOPOGRAPHIE DIGITALE」、「TIMELINES」、「DIVING INTO AN ACIDIFYING OCEAN」、「ETHEREAL FLEETING 」

作者：[Audrey Briot, Martin De Bie, Alice Giordani, CédricHonnet]、[Fabian Oefner]、[Fabian Oefner, Cristina Tarquini, Laurie Frick, Pekka Niittyvirta, Timo Aho]、[Lukas Truniger, Itamar Bergfreund,Bruce Yoder]

參考資料來源：

TOPOGRAPHIE DIGITALE—

https://ars.electronica.art/keplersgardens/en/topographie-digitale/

http://datapaulette.org/work/topographie-digitale/

TIMELINES—

https://ars.electronica.art/keplersgardens/en/timelines/

DIVING INTO AN ACIDIFYING OCEAN—

https://ars.electronica.art/keplersgardens/en/acidifying-ocean/

ETHEREAL FLEETING—

https://ars.electronica.art/keplersgardens/en/ethereal-fleeting/

作者：Akira  Nakayasu

主題 : 中安翌 Nakayasu 相關作品

參考資料來源：http://nakayasu.com/

透過了解，生物與植物之間的日常活動中不起眼、不被注意的部分，去解析且再用SMA來作出再表現的運動作用，用加熱伸縮的材料特性，去作出延伸的運動方向，而靈感就是來自地表上的植物，或是海洋的海葵，透過自然力而隨著搖擺，再加上科技層面的部分，類似聲音、光等部件結合成一項完整的作品，擷取觀眾的互動來應應作品達到一種美好的關聯，而使用形狀記憶合金運動 ( SMA )顯示技術來表達逼真的動作，將生活中所沒意識到的、常態的小事情透過不同的手法在表現，就會非常地吸引人。