Apple's Augmented Reality
Rumours about Apple's Augmented Reality (AR) development plans are popping up often last months. There are strong suspicions the new iPhone 8 will be armed with AR gear like advanced 3D camera's and the smart connection at the bottom that will enable wireless charging and, in addition, the same connection will be used by the new iPhone users in the VR / AR experience which Apple has been working on lately. Also Developer Steve Throughton-Smith discovered iOS 10.3 (beta) has the possibility to handle more than 60 frames a second.
This could be intended for AR and VR Applications. 90 fps is considered a minimum for good immersive VR and AR experiences.
Apple is not the only one looking into the possibility of combining smart phones and AR. Companies like Occipital and Meleap also have big plans to launch AR enabling products for Apple´s iPhone. The Occipital Bridge consumers version will be released this month.
The Merge Cube (formerly called HoloCube)
The Merge Cube, which lets users interact with holograms through augmented reality (AR) technology, was first presented at this years CES along with a lot wide range of colored Merge VR headsets.
Merge Labs describes the HoloCube as the world's first interactive, holographic toy that you can hold in the palm of your hand.
“Toys are the tools we use for growing up, and our HoloCube offers brand new ways to learn, play, and connect with others. Rather than using the typical interfaces of 2D screens, we’re developing physical products that merge the real world with the digital, and creating new ways for imagination and creativity to flourish.”
Merge VR Founder Franklin Lyons @ CES 2017
The Merge Cube is a lightweight foam cube with a unique print on each of the cubes sides. It is the same material used to create the Merge VR headset.
Merge sent me a development kit of the Merge Cube to test and develop.
After receiving the Merge Cube dev-kit I started by downloading the sample App to test the Merge Cube. It's a very simple demo App with 3 holographic experiences:
A cube in a cube, a soundbox with a button to switch songs and a with fireworks.
Enough to understand what capabilities and possibilities the Merge Cube has to offer.
For those who are not familiar with Augmented Reality (AR) a brief description:
"Augmented reality (AR) is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data."
The first widely known AR prototype was the Google Glass, which did not succeed in attracting a broad fan base. Google Glass created a 2D overlay with information on top of the real world. The third version is beeing sold by Google inc on Amazon and similar websites. The Microsoft Hololens is the second big attempt to bring AR to the masses. But this time its a headset that enables 3D virtual objects to be projected into the real world. This is also described as Mixed Reality. At the moment of writing the Hololens is intended for development purposes and costs between 3000 and 4000 dollars on Amazon. Its obvious this product will not be a mass consumption product but it might be the Oculus Rift of the AR headsets. There are many companies developing similar glasses.
Back to the Merge Cube:
Merge Cube compatible Apps project 3D virtual objects on the Merge Cube when looking through the screen of your iPhone or iPad.
The gadget works best if used together with a AR capable VR-headset and a iPhone (preferably iPhone 7 plus).
The Merge VR, Merge its own headset, is AR-capable and headsets like Homido, 2VR, Google Cardboard v2, I Am Cardboard Pocket 360 and many others also do the job. AR-capable headsets are VR-headsets that allow the camera of your mobile phone to be used.
You can also use a iPhone or iPad without a VR-headset, depending on the App/Game. It works but it´s just not the same.
I went a step further and tested the Merge Cube in combination with a Wide-angle lens and this improves the AR experience al lot. The regular FOV is very limited. The Wide-angle lens gives a much wider FOV. For this experiment I used the Occipital Bridge headset which comes with a high quality Wide-angle lens and is specially created to fit an iPhone 6. The Structure Sensor, which usually is combined with the Bridge headset, is not necessary in this case. There are cheaper options on the market that achieve the same. In theory any iPhone compatible Wide-angle lens (like this one on Amazon) in combination with the 2VR headset should also do the trick. I havent tested this yet. Keep in mind that for this to work you need a VR-headset with enough space to add the Wide-angle lens and that is AR-capable.
Developing a Merge Cube experience
After playing around with the demo app, which one must build yourself with Xcode, I decided to create a demo game myself.
To create content Merge offers a free SDK and another SDK called Vuforia is also needed. Import both in Unity3D, import content (like 3D models, sounds and physics) and after a few button clicks a Xcode project is ready to be built and loaded into a iPhone.
So here's my humble creation. A 3D Ball-in-a-Maze Game. Simple fun. But it clearly shows the possibilities the Merge Cube offers.
A big difference between the Merge Cube AR experience and Mixed Reality devices like Merge Lens is that with the Merge Cube the AR-experience is limited to the Merge Cube itself. The HoloLens can project AR-experiences throughout (almost) your whole Field of view onto any surface. With a very higher price tag.
A combination of AR-glasses and the Merge Cube would expand possibilities even further.
Downside of the whole Merge Cube concept at the moment is that, in order to develop for the Merge Cube, you need a 99$ (a month!) Vuforia license per App/Game.
Most Indie Developers won't willing to pay such amount for putting a niche App in the AppStore. This might limit content creation.
So For those who want to develop for regular / non-VR experiences Merge offers a SDK that can be used without the Vuforia license. I hope there will be a similar option for VR content creation in the near future.
Available Apps (update 10 November 2017)
Country of origin: United States