Apple is serious about augmented reality. Tim Cook has been clear about how important it is, saying it will “change the way we use technology forever.” With ARKit in iOS 11, Apple brought AR capabilities to hundreds of millions of phones. With iOS 12, Apple brings developers ARKit 2 with enhanced capabilities like 3D object detection, persistent experiences, and shared AR spaces.
It’s a robust improvement on what is arguably the best set of augmented reality tools for mobile devices anywhere.
What’s new in ARKit 2
When ARKit debuted with iOS 11 last September it immediately made waves, but the limitations were obvious. With iOS 11.3, the ARKit developer tools got more powerful, adding the ability to track vertical surfaces and non-flat surfaces. Now, just one year later, Apple is updating to ARKit 2. Here’s what Apple says is new:
Improved face tracking – Apple didn’t dive into this, but the name is self-explanatory.
Realistic rendering – The rendering of augmented reality objects is already quite good. Apple didn’t say exactly how rendering has improved, but we expect more accurate lighting is key.
3D object detection – ARKit detects flat rectangle objects like posters or book covers, but ARKit 2 lets developers detect fully 3D objects. That’s going to be huge.
Persistent experiences – You can save AR spaces and objects that are linked to physical objects (like toys) or physical spaces (like classrooms), so you can pick up where you left off later.
Shared experiences – Multiple users can use their iOS devices to view the same virtual environment, each from their own view. Apple is releasing a block-breaking multiplayer game as a code demo, and Lego showed off a virtual playspace where up to four players can interact with a combined virtual and real play space all at once.
IDGLEGO showed an impressive demo that used nearly all of ARKit 2’s new capabilities: 3D object detection, multiplayer, and persistence.
Measure app
When ARKit launched, it was quickly followed by a flood of measurement apps. Apps to take virtual measurements of lengths, volumes, room sizes, you name it. Apple is getting into the game with its own measurement app, aptly named Measure.
IDGMeasure lets you make linear measurements of real-world objeccts, along all three axes.
Measure looks like a simple, reliable version of the many AR measurement apps out there. You can make linear measurements, even of 3D objects to quickly compute volume. It even recognizes rectangular objects automatically, so you just point at a poster, photo, or rectangular table, tap, and get a full set of measurements.
IDGMeasure automatically recognizes rectangles, so you can measure with a single tap.
This is one of Apple’s simpler apps, like the Calculator, Compass, or the built-in Flashlight. And like those apps, it’s an appropriate thing to have built into your phone, but not the sort of groundbreaking experience that will make people switch to iPhone.
USDZ format
Apple worked together with Pixar to develop a new file format for augmented reality objects called USDZ. It’s going to be supported by third-party apps, including ones by Autodesk, Adobe, Sketchfab, and more.
IDGThe USDZ format was shown integrated into Fender’s website, allowing you to see your customized guitar in the real world.
Adobe CTO Abhay Parasnis took to the stage to say that Creative Cloud will support the format, and that their tools will even allow what-you-see-is-what-you-get editing of VR objects with iOS devices. Does this mean that USDZ will become a widely supported format, even beyond Apple devices? Apple wasn’t clear, but it looks like the odds are good that USDZ could becomes the next HEIC.
Supported devices
Per Apple, “ARKit 2 and usdz support will be available this fall as part of a free software update for iPhone 6s and later, all iPad Pro models, iPad 5th generation and iPad 6th generation.” That’s pretty much every device that supported the original ARKit release.
Author: Jason Cross, Senior Editor
I have written about technology for my entire professional life - over 25 years. I enjoy learning about how complicated technology works and explaining it in a way anyone can understand.