Designing Sound Representations for Responsive Environments

Designing Sound Representations for Responsive Environments

Designing Sound Representations for Responsive Environments

センサーとコンピュータを使用したインタラクティブインスタレーションを展示しました。コンセプトは多くのデータを出力する温度センサーとカメラ画像からの音と映像（場面）を使った新しいピクトグラムを作る事、具体的には温度と場面の特徴をつかみ音や映像で簡略化（簡単な記号として表したピクトグラム）し表現する事です。温度差がある屋外と屋内の住居にそれぞれ、センサー、カメラを設置し、外温と室内温を層化したサウンドの素材として使い、これら場面内のカメラ像の人の動きや状況を同時にピクトグラムとして表現する事で、屋外での出来事と屋内の出来事（離れた距離の出来事）が、同時に音と映像で把握できる事、また、建物に出入りする人が音の層を横切るように連続した音の変化でその情景を理解できるようにデザインをしました。特に、季節の変化などの長期的な気温の変化、日夜の音の変化（色々な時系列で音と映像の違いを感じる事）がわかり、時間を感じとれるピクトグラムを構想し、これらの応答装置を日常利用する変化する装飾家具や置物のように感じてもらうことを望んでます。また、このデモンストレーションのためにシステムを実装しました。３台のウェブカメラと３つの温度計をつなげ、温度と動きの出力データと連動させた信号音と背景音生成し、CGの画像へと反映させました。３カ所でカメラと温度センサーを設置し、動体検出と温度測定の結果をOSCプロトコルにて、音響生成エンジン（Max/MSP）とプロジェクション（Processing）解析を行うユニットへと送信します。音響生成エンジンでは動体と温度によるサウンドをリアルタイムで生成する。プロジェクションにおいては温度により変化するピクトグラムをプロジェクターによって投影させています。

Herein, we demonstrate a method for creating a sound installation that uses sensors and computers to present computer-generated sound and novel pictogram imagery using information obtained from cameras and thermometers. The computer-generated pictogram sound and image data are presented in a different location from the collection point, and the audience can recognize the changing states by listening to the sound and/or watching the imagery. The system uses thermometers and cameras that are installed in both interior and exterior spaces to detect the state of those spaces based on the obtained image and temperature data. When the state of the captured images or the temperature changes, such as when a moving object is detected, a signal tone and a pictogram are created and assigned to describe the state based on a temperature and image data-mapping table. The designed sounds change according to the detected states, and the proposed installation simultaneously produces sounds and pictograms in different spaces in order to convey information related to motion or temperature states. For example, if the temperatures of the interior and exterior spaces are different, different sound pitches are assigned to each space. Thus, individuals walking from outside to inside, or vice versa, will experience a change in sound. If the installation were demonstrated in both interior and exterior spaces, the temperature in the spaces would also change with the season. For example, if the installation were demonstrated in winter, the inside space would be hot, and outside would be cool, or vice versa in summer. If the installation is considered from a long-term point of view, individuals can experience sounds of varying pitch, which change with temperature, as the seasons change. Consequently, the assigned sound provides information on both motion concerning a moment and temperature regarding a long period of time.

Hardware thermometer, projector, speaker and camera
Software Max/msp, processing

Hardware thermometer, projector, speaker and camera
Software Max/msp, processing

http://www.icad.org/icad2016/proceedings2/papers/ICAD2016_paper_2.pdf

https://vimeo.com/172741839

http://yinteraction-design.com/event/

Takuya Yamauchi (Software Developer, Media Artist, Interaction Designer and UX Designer) Ph.D, Graduate School of Media and Governance, Keio University (2009)

(Exhibition)
“Tokyo”, Art Gallery, SIGGRAPH ASIA 2020
“River’s Edge”, Art Gallery, SIGGRAPH ASIA 2020

(Journal)
1 "Design of a process for interactive product in ubiquitous space " Takuya Yamauchi and Toru Iwatake International Journal on Interactive Design and Manufacturing (IJIDeM) Volume 2, Number 2 / 2008,5 Springer Verlag
2 "Sound Jewelry" Takuya Yamauchi and Toru Iwatake Leonardo Music Journal No 18 MIT Press
3 "Design of a process for Localization System in ubiquitous space " Takuya Yamauchi and Toru Iwatake International Journal on Interactive Design and Manufacturing (IJIDeM) Springer Verlag
(Conference)
1 "Designing Sound Representations for Responsive Environments" Takuya Yamauchi, The 22nd Annual International Conference on Audiotry Display (ICAD2016),Australian National University, Canberra 3-7 July 2016

Office Yamauchi 150-6090 Ebisu 4-20-4 Shibuya-ku Tokyo Japan Ebisu Garden Place, Portal POINT Ebisu E-mail: yamauchi@yinteraction-design.com

日本