What is this project for?
On October 31st 2019, Shuri Castle was destroyed by a devastating fire. People all over the world are shocked by this news and we are also deeply saddened. We are researchers, students, and engineers who are working on problems surrounding Computer Vision (CV), Human-Computer Interaction (HCI), Virtual Reality (VR), and other related problems. We started this project in hopes that we can do something to help restore this important piece of cultural heritage, and cheer up others who were also saddened by this tragic incident.
In the field of computer vision, which aims to design computer algorithms to process and understand images, there are technologies that can reconstruct 3D shapes and textures of objects using images and videos. Those can be utilized to preserve pieces of cultural heritage. One example of such project is eHeritage.
Having heard the news of the fire, as people working on computer vision, we hope that we can help to recover these touristic resources by reconstructing a 3D model of Shuri Castle, until the parts that were damaged in the fire are rebuilt. The video above shows the concept of our project. The 3D model in this video is reconstructed using public software from the images we collected by image searching. If there are more photos and videos with a lot of diverse viewpoints, we can create a much more beautiful 3D model. Right now, our dataset is limited and thus our reconstruction is also imperfect. We want to utilize your photos and videos to create a better 3D model and share your memories of Shuri Castle with others who are also saddened by the news. Please share your photos, videos and memories with us for a digital reconstruction of Shuri Castle.
What can we do with the photos / videos collected and the memories shared? We can reconstruct more precise and complete 3D model with more data. With the contents created by the photos / videos, you can see the lost part overlapped on the local landscape through a smart glass or a mobile phone in Okinawa. You can also revive the memories of the past of other people at the place where he/she has been by using the shared memories. With a head-mounted display, you can have the same experience even if you are not at the site in Okinawa. Also, with 3D models reconstructed for each era, you will be able to experience the Shuri Castle and its change over time.
Please share your memories of Shuri Castle!
Why are we working on this?
We are volunteer members who were shocked and heartbroken when we saw the fire news. We felt that we should do something socially meaningful for Shuri Castle with 3D reconstruction technology that is closely related to our expertise, and making this happen by collecting data from everyone who have memories of visiting Shuri Castle. The loss of Shuri Castle is disastrous, but a 3D model restored with everyone’s data will encourage people and allow the castle to remain in their hearts for a long time. Fortunately, we are a group of people with varied specialties. We decided to partake in a project that has good social impact with ideas shared by the members.
In the field of computer vision, there is a technology called Structure from Motion (SfM). This is used to reconstruct 3D shapes using photos and videos from different viewpoints. Our reconstruction of Shuri Castle also uses this technology.
It has already been decades since the early ideas for 3D reconstruction, including SfM, were published, and continuous effort has been made by outstanding researchers to progress the field. Now, thanks to the contributions of great engineers, the technology has developed to a point where we can easily create 3D reconstructions using our computers as the 3D model as shown in the video above. During the International Conference on Computer Vision 2019 (ICCV 2019), Building Rome in a Day, a seminal work on SfM, was awarded the Helmholtz prize for its long term impact in the field of computer vision. This was 2 days before the devastating fire broke out in Okinawa. This work addresses various scalability issues which arise from trying to reconstruct a 3D model from tens of thousands of images. This work allowed the digital reconstruction of the Colosseum of Rome and the city of Dubrovnik from images from the internet.
Several members of our team have participated in ICCV 2019 for their own research, and several of them were the organizers of a workshop for digital preservation of cultural properties at ICCV 2019. When one of the organizers saw the news of the devastating fire, she wondered what she could do to help. When she called out to the researchers and students around her, she got great support and many members have devoted their time to this project since then. After that, we have been discussing what we can do to contribute to society by creating something that will remain in everyone’s hearts.
Why do we spend our work / study time doing these activities? Because we are glad if it helps people. Not only that, it is a pleasure to let people know the utility of our field through this kind of cooperation. Our member’s research areas are computer vision, human-computer interaction, virtual reality, machine learning, and more. We are very happy if we can make you interested in the work we do.
- Rei Kawakami (Tokyo Institute of Technology/Denso IT Laboratory)
- Wen Shao (The University of Tokyo)
- Ikuo Kamei (The University of Tokyo)
- Hajime Itsukaichi (The University of Tokyo)
- Takeshi Naemura (The University of Tokyo)
- Towaki Takikawa (University of Waterloo)
- Takefumi Hiraki (Osaka University)
- Yota Egusa (SAKURA internet Inc.)
- Eduardo Acosta (Raiz New Media)
- Raul Acosta (Raiz New Media)
- Karin Kiho (The University of Tokyo)
- Han Changyo (The University of Tokyo)
- Yuichi Miyamoto (Party Inc.)
- Yuzuko Utsumi (Osaka Prefecture University)
- Katsushi Ikeuchi (Microsoft)
- Jean Ponce (INRIA / New York University)
- Jonathan Chemla (Iconem)
- Yves Ubelmann (Iconem)
- Miku Sega (gluon)
- Daiki Sugiyama (UmeeT)
- Daisaku Kina (Ryukyu history researcher)
- Ryo Fujiwara (HoloLab Inc. / xRArchi)
- Yasutaka Furukawa (Simon Fraser University)
- Yasuaki Kakehi (The University of Tokyo)
- Wataru Date (Graphic Designer)
- Furuta Katsumi(Keio University)
- Martial Hebert (Carnegie Mellon University)
- Rujirat (Vinitphol) Ishikawa (The University of Tokyo)
- Marin Kamizono (Graphic Recorder)
- Marc Pollefeys (ETH Zurich / Microsoft)
- Johannes Schöenberger (Microsoft)
- Yuto Kodama (The University of Tokyo)
- Yinan Wang (The University of Tokyo)
- Pai-Hsien Hung (Osaka University)
- Yi-Jhen Lin
- Tasuku Mizuno (CITY LIGHTS LAW)
- Changchang Wu, “Towards Linear-time Incremental Structure From Motion”, 3DV 2013
- Changchang Wu, “VisualSFM: A Visual Structure from Motion System”, http://ccwu.me/vsfm/, 2011
- Changchang Wu, Sameer Agarwal, Brian Curless, and Steven M. Seitz, “Multicore Bundle Adjustment”, CVPR 2011
- Changchang Wu, “SiftGPU: A GPU implementation of Scale Invaraint Feature Transform (SIFT)”, http://cs.unc.edu/~ccwu/siftgpu, 2007
- Yasutaka Furukawa, Brian Curless, Steven M. Seitz, and Richard Szeliski, “Towards Internet-scale Multi-view Stereo”, CVPR 2010.
- Michael Kazhdan, Hugues Hoppe, “Screened poisson surface reconstruction”, ACM Transactions on Graphics (TOG), Volume 32 Issue 3, June 2013
- P. Cignoni, M. Callieri, M. Corsini, M. Dellepiane, F. Ganovelli, G. Ranzuglia, “MeshLab: an Open-Source Mesh Processing Tool”, Sixth Eurographics Italian Chapter Conference, page 129-136, 2008
- Sameer Agarwal, Yasutaka Furukawa, Noah Snavely, Ian Simon, Brian Curless, Steven M. Seitz, and Richard Szeliski. 2011. “Building Rome in a day”. Commun. ACM 54, 10 (October 2011), 105-112.
- Y. Furukawa and J. Ponce. “Accurate, Dense, and Robust Multi-View Stereopsis”. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8):1362-1376, 2010.
- Johannes L. Schönberger and Jan-Michael Frahm. “Structure-from-motion revisited.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
- Johannes L. Schönberger, Enliang Zheng, Jan-Michael Frahm, and Marc Pollefeys. “Pixelwise view selection for unstructured multi-view stereo.” European Conference on Computer Vision. Springer, 2016.
This project is supported by SAKURA internet Inc.
This website uses “Sakura Cloud” and “Sakura Web Accelerator“.
This website uses 3D viewer developed by Sketchfab, Inc.
This project is supported by CapturingReality, developing 3D reconstruction software RealityCapture.
This project uses image archive collected by Okinawa Times.
This project uses Azure environment supported by Microsoft Japan.
This project uses images collected by Konan Act Club at Konan high and Junior high school.
This project’s image collection is supported by Yokohama Fujimigaoka Junior high and high school.
This project’s image collection is supported by Baidu Baike and Baidu Baike Museums Project.
This project is supported by Okinawa Prefecture.