visionOS

• Full Liquid Glass Make it easier to write cross-platform apps by using glassEffect everywhere.
• A Camera in RealityView I would love to be able to add a camera within a RealityView and be able to record or render that view elsewhere. As a source for a PortalComponent could also be very interesting.
• ARKit body detection of the wearer having more of the skeleton of the visionOS user would unlock a lot of exciting game and health use cases
• Widgets in environments I want to leave a widget in a specific place on the Moon, or on the surface of Haleakalā, and have it still be there next time I'm in that environment.
• Even more flexible widget placement and window pinning I have objects on my walls that interfere with widgets, and I'd like to use any spot along that wall regardless of what's physically there. One idea: if I shake the widget a lot, or hold it in place long enough, it breaks through the restrictions and gives me a more custom positioning that reflects the wall itself instead of the room scan.
• Shared coordinate spaces between iOS and visionOS We have shared in-person experiences for visionOS to visionOS, and for iOS to iOS, but not crossing that bridge. I have a lot of app ideas that would be unlocked with this, which is why I've been working to get this going manually, but I would love to get native support in iOS 27 and visionOS 27.
• visionOS and tvOS more automatic media synchronization I'd like to start watching something on the TV and in Vision Pro at the same time, with playback controls synced automatically.
• AirPlay by gesture Perhaps some gestures to point at a device in space to start casting to it.
• Spatial templates and SharePlay for more than five participants I want to add "flat" FaceTime participants from iOS, macOS or the web to an immersive template and give them spatial positions in the scene, instead of grouping them on the shared canvas. Literally let me place people around a virtual table, whether they're Spatial Personas or flat video heads.
• ARKit information from immersive scenes Let me place virtual content in the user's selected immersive space as if it were a real space they're standing in — the contours of the volcano, the craters and rocks on the Moon.
• Render and interact with a captured space Gaussian Splats perhaps — with an associated mesh exposed as ARKit anchors so I can treat it like a real space for placement and physics.
• "Leave behind" objects into the shared space I want to offer that a user can take elements out of my apps and set them on a table, a bookshelf, wherever, where they remain interactive as new windows of my app in the shared space. Think of this like tear off toolbars from the Mac. I want to be able to "pin" something from my mixed immersive experience to the real world and have it stay in the Shared Space when I leave.

RealityKit

• More pre-made assets — objects, textures, sounds, everything. Please bring the models from the old Reality Composer app to current USD and more.
• More lights, and more lighting options More lights at one time in a RealityView scene, and more lighting effects. I'd love to be able to have light masks so they cast unique patterns, animate them, and more.
• Live previewing of Components and Systems inside Reality Composer Pro Let me test and run my Systems in the same preview environment where I'm authoring the scene.
• Precise camera control - I would like to be able to specifically position the camera and its focal properties in RealityView.
• Immersive on iOS and macOS - I would like some equivalent to Immersive Spaces on iOS and macOS. To go full screen into a 3D view, treating the screen as a portal you are looking thru could be a really compelling way to bring visionOS experiences on iOS and macOS.

Reality Composer Pro

• Dramatically faster compile times Builds come to a long stall compiling rkassets. Any performance improvements here would be very welcome.
• Baked lighting If I could mark lights or objects as static in an RCP scene and have that lighting baked into the textures at save/load time, scenes would look dramatically better with no extra runtime cost

Foundation Models Framework and CoreAI

• I would like to see some built-in tools to connect to Foundation Models Framework that provide access to information and context about the user (with their permission of course), to make for more capable and interesting combinations of personalized reasoning for users.
• There is a lot more to explore with on-device reasoning and I'm really excited to see where things expand beyond Foundation Models Framework.
• I've seen rumors of a CoreAI SDK, and would be excited to see what additional tools Apple might bring to provide developers with more powerful reasoning.

MapKit

• Camera control with 3D and street level views - I want to be able to animate a camera moving along a street with the street-level perspective and views. Imagine a leisurely walk down a street on any device.

ARKit

• Cross-platform I want more aligned sensor data between iOS and visionOS
• Multiple body detection I would like to be able to detect and track multiple figures and body skeletons and get 3D positioning for all of them. Bonus if this uses the ultra-wide camera

Miscellany

• Gaussian Splat SDK My dream would be a full SDK with tools for creating / editing / combining splats into RealityView scenes.

That's the list. Some of it is incremental, some of it is a big stretch, and hopefully at least one or two items will show up in June.

With the recent 50th anniversary for Apple, I've been reflecting back a lot, and Cintos Mobile was my first ever iPhone app. I'm glad it's around again.

Before the App Store, the iPhone shipped with no SDK

The first iPhone came out in June 2007. There was no App Store. There was no native SDK for third-party developers. If you wanted to build "an app" for the iPhone, the official answer from Apple was: build a web app.

At WWDC 2007, Steve Jobs framed it pretty directly: developers should build for the iPhone using Web 2.0 and AJAX. The pitch was that those web apps could look and behave like the native ones, integrate with iPhone services like Mail and Maps, and ship without an SDK at all — your existing browser-side skills were everything you needed.

That was the official path. So that was the path I took.

The App Store wouldn't be announced until March 2008 (with iPhone OS 2.0 and a real native SDK), and wouldn't open to the public until July 10, 2008. In that roughly year-long window, "iPhone apps" meant Safari + viewport meta tag + a lot of -webkit- CSS.

Cintos Mobile

Cintos was a "board game" I made first for the Mac, and I used it as a way to practice as I explored other languages or platforms. I had made a Java version and had toyed with a Windows version too. It scaled well to the iPhone screen with the 8×8 board across the 320×480 screen. The whole thing is HTML, JavaScript, and a few PNG files.

If you want to play right now: /cintosmobile/.

I tried out the web app at my nearest Apple Store once it was online, and made a few trips back until it was working right.

A few details from the source that look very much of-their-time:

<meta name="viewport" id="iphone-viewport"
      content="width=320, user-scalable=no" />

width=320 because the iPhone's screen was 320 points wide. user-scalable=no to prevent pinch to zoom from messing up the view.

<style type="text/css" title="AppleStyle">
    @import "cintosmobile.css";
</style>

title="AppleStyle" because I started from some Apple code.

After the App Store

When Apple announced the App Store and the native SDK, I applied as soon as I could and was accepted into the iPhone Developer Program early. My first App Store app was Flashlight, which ended up being downloaded over 100 million times.

A note on the restoration

The HTML, JS, and CSS are pulled directly from the Internet Archive's Wayback Machine — specifically from a January 2009 snapshot, with a few small edits: the "back to website" button now points at this site rather than the old one, and three game piece PNGs (piece0.png, piece1.png, piece2.png) had been lost from the archive entirely, so I drew new ones at the original 32×32 dimensions to make the board playable again.

Other than that — same code, same layout, same year-2007 web-app energy. :-)

Here are my basic results compiling a template project for visionOS which includes an Immersive Scene:

All of these tests were using Xcode 26.4 beta 3

Device	Mac Virtual Display	Compile Time (seconds)
MacBook M4 Air	No	22.5
MacBook Pro M1 Max	No	26
MacBook Neo	No	157.5
MacBook Neo	Yes	372.5

I would say the MacBook Neo does a fine job for everything, and if this is your first Mac. At $499 for education, I'm very pleased that this can handle running the visionOS simulator and everything else. Happy coding!]]> https://johnhaney.com/blog/manipulation-on-visionos/ https://johnhaney.com/blog/manipulation-on-visionos/ Fri, 13 Jun 2025 00:00:00 +0000 Of all the great things introduced at WWDC25 this year, ManipulationComponent and the manipulable() SwiftUI modifier are really great. These provide some rather natural interactions with objects with one line very little code.

I just published an open-source sample visionOS app, Manipulatives, which demonstrates the basics of this. My approach is to group everything I want to become manipulable as a child of one Entity with a specific name. Then I can search for that parent Entity, and loop thru all of its children adding ManipulationComponent. This would, of course, be even simpler if the ManipulationComponent were available directly form within Reality Composer Pro (Feel free to file a Feedback and reference my FB17979086).

With ManipulationComponent and Entity Observables, I see this as a great combination to tracking how users are moving entities around. I’m excited to put this into action and look at ways to turn user’s manipulations of Entities into dynamic behaviors and actions in my apps’ scenes.

Sometimes we get major new announcements, like visionOS and the Apple Vision Pro at WWDC23.

I have been making apps for visionOS ever since and I’m really excited about where this platform is heading.

My wishlist for visionOS

• Supercharge the Shared Space
  - I want apps to be able to leverage knowledge of the users hands when they are inside a volume in the shared space. Think of making a drum set app where you can position each drum as a volume just where you want it, and then play in the shared space while you have your music up in another app, and maybe be recording in GarageBand in another window
  - FB16447897 (visionOS Hand Anchoring Component in Shared Space (Volumetric Windows))
  • As a developer, I want to offer “pinning” hints to visionOS. I want to be able to suggest that this window could be pinned to a table of minimum width/depth or to a wall or next to a TV, …
  • I want to allow ARKit sessions to run within the overlap of my volumetric windows in the shared space. Let me “see” the mesh or table or wall that the volume overlaps with. Let me see the relative position of my open windows / volumes in the shared space. With this, I could make an app where you place a volume in opposite corners of a room and get the dimensions quickly.
  • As a user, I want to make a mapping of my home and name the rooms, ideally maybe even those are picked up and/or synced with HomeKit. Then allow me to share that with developers in a privacy preserving way so that I can configure apps to work differently in different rooms as I move around.
  • I’d like to be able to pin multiple instances of an app to different rooms and have the app available to me when I am in the room, and dismissed when I leave the room
  • Native development tools on visionOS - Reality Composer Pro, Xcode, Swift Playgrounds
  • Shared in-person sessions where each visionOS device has a common anchored position for content. To interact in a mixed immersive space with others would unlock some great experiences
  • Shared in-person sessions between visionOS, iOS, macOS. With RealityView on all of these platforms, having a common coordinate space would again unlock some powerful interactions
  • Spatial Templates for any number of participants (visionOS or not). I want to be able to provide “seat” assignments to everyone whether they are sharing a Spatial Persona or not, and whether they are even on a visionOS device or not. This would allow me to place people at “tables” for a conference session, or give proper assignments for a game night
  • TabletopKit for iOS and macOS also. Extending this framework to any platform that has RealityView support seems natural to me.
  • FB13873238 (TabletopKit support for iOS, iPadOS, macOS, tvOS)

One more thing…

I’d like to give my aspirations for where I hope Apple might be headed. I’m sure that I’m wrong, but I want to put out there what I would like to see and cast my own view of the future out there.

The next big evolution I see is around Spatial Computing. I want to see Apple combine the awareness of where I am in space (Center Stage) to turn the flat displays of iPhone, iPad, Mac, AppleTV into portals into immersive spaces. If I use my iPhone with an AppleTV I should be able to use Continuity Camera to enable this same tracking on a big TV.

With a full awareness of my position relative to the display, the display now becomes a view in space that I can move my head around to explore and watch it respond in real time.

Maybe I can use an Apple Pencil Pro to poke into the space for more precise selection.

Then I want a truly Spatial Home Screen and work environment. When I open my MacBook at my office desk, my work environment is ready for me, with slack tucked away to the side. I can “look around the corner” to pull out the music mini player and manage other glanceable tasks.

Here's what it looks like in action:

Xcode keeps themes in a folder: ~/Library/Developer/Xcode/UserData/FontAndColorThemes so here are my recommended steps to add this theme to your Xcode:

1. Open Xcode

2. Choose Xcode > Preferences

3. Choose Fonts & Colors

4. Press the + button in the bottom left corner and make a new theme named "Mulder"

5. Quit Xcode

6. Copy the downloaded Mulder.dvtcolortheme file to ~/Library/Developer/Xcode/UserData/FontAndColorThemes/.

7. Open Xcode and enjoy a greener coding experience. :-)

Suddenly, a fox appeared in front of my car on the left-hand side. I swerved the car to the right off the edge of the pavement with right-side tires onto the gravel shoulder, and turned back to return to the road. The car started skidding with the rear moving to the right. I recalled some old driving test television shows I watched in my early teenage years and knew I was supposed to take my foot off the gas and turn into the skid, and to be careful not to overcorrect. I took my foot off the gas, I turned into the skid -- too much. The car continued skidding, this time off to the right-side of the road. I saw an overpass ahead around the corner. I saw a car coming from the other direction. Then I saw grass as the car turned toward the rising hill off the side of the highway and my car skidded further off the edge of the road. I didn't really have time to even process it, but I was glad to be moving away from that car.

The next thing I knew the car was upside down. I was suspended by my seatbelt and didn't want to be like that. I put a hand on the roof of the car to brace myself as I unbuckled and rolled myself out of the car. My door opened perfectly, the window still whole in fact. The windshield had shattered. Every other window was shattered. The car was upside down, over the guardrail it had just smashed thru. It wasn't until I got the police report and saw the officer's sketch that I realized how exactly the car got to that position. The driver's side door behind me was demolished. I can't imagine any person sitting in that seat surviving the crash. I was a foot away from that. Had I been driving a bit faster, it's just as likely that I would've hit the guardrail at my door instead of the empty one behind me.

I was impressed and grateful for all of the people who stopped and helped to call 911. I'm especially thankful for the police officer who found my phone and glasses among the belongings that had flown around in the wreck.

A colleague of mine told me about a belief in China that surviving an experience like this one meant I was marked for something great. I've keep considering this. I want my life to be worthy of this gift. This borrowed time that I'm living on since that day. My family is the legacy I care most about, and I cannot be prouder of what Katie and I have grown. All of our children are so smart, kind, playful, and empathetic.

I was fortunate enough to be one of the first developers to make apps on the App Store, and my Flashlight app has been used by millions of people. That's still so surreal for me to think about, and I'm so thankful for the opportunity to share my work with so many people.

I've had 13 years since this crash, and I've thankfully been able to do longer road trips without thinking about this. Some things certainly fade with time. My wife's birthday is November 13th. My daughter's ultrasound was on Friday the 13th.

13 is my lucky number.

Thanks again! Now it's time to get a little rest… :-)

This includes my tips on where/when to do certain actions to balance resource usage and keep your UI snappy.

Here is the PDF of the presentation: CocoaHeads ViewController Talk

I'm happy to report that the new Daily Challenge Server is up and running, and my initial testing is going very well. Soon (within the next week or two), CryptoQuote for iPhone, iPod touch, and iPad will be updated to version 2.4. This version is designed to use the new server.

Once I'm happy with the iOS update running "in the wild," Android apps will be next.

So, in a few weeks, CryptoQuote for Android will be updated to version 2.0, adding the Daily Challenge feature for Android users, including Kindle Fire, NOOKcolor, and NOOKtablet.

Greetings,

I'm very excited to share with you that CryptoQuote is now available for some Android devices (tablets with access to the Amazon App Store, including Kindle Fire). You can download CryptoQuote for Android 1.0 here.

This version is launching with many of the great features of CryptoQuote you see on OS X and iOS as well, with a few to be added in the not-so-distant future.

At this moment, CryptoQuote is available solely via the Amazon App Store. My hope is to have the app available for NOOK devices within the next few weeks (depending upon the pace of Barnes & Noble approval processes).

An Apology
The initial description on the Amazon App Store product page for CryptoQuote 1.0 includes the following line:

Features: * 3199 carefully selected quotes, plus a new Daily Challenge Puzzle each day

About the Daily Challenge
The Daily Challenge feature is currently only available for iOS devices. Please do know that I am committed to bringing the daily challenge to Android devices and Mac OS X. Please also know that this is not without some significant steps before this can happen. I'd like to elaborate a bit on this…

The current daily challenge server needs some improvements:

1. It is currently on an older host system, which shares resources and power with other websites.
2. It does not respond quickly enough to requests.
3. It does not handle lots (and lots) of iPhone users well.
4. It gets slower as more people play the daily challenge each day.

My plan is to address these problems first for iPhone users, by doing the following:

1. Rewrite the server-side code to handle requests quickly and efficiently.
2. Restructure the database to handle requests quickly even for lots (and lots) of users.
3. Setup this new server code on my new host machine, and migrate the daily challenge users over to the new server.
4. Update CryptoQuote for iOS to add the Daily Challenge.
5. Update CryptoQuote for Android to add the Daily Challenge.

Questions or comments?

Contact me via: http://appsfromouterspace.com/contact/
or on Twitter: http://twitter.com/johnhaney

I'm calling this my online moving day. Here's what's happening:
Twitter:
@JohnHaneySW -> @Apps_OuterSpace

Facebook:
facebook.com/JohnHaneySW -> facebook.com/AppsFromOuterSpace

Website:
johnhaney.com -> appsfromouterspace.com

App Store:
Coming Soon. I'm awaiting a few finalizing details. For the moment, apps will be listed under the "John Haney Software" name.

So please update your contacts, likes, follows. 2012 is going to be a great year, starting with my next app, Puppet Stage. But I'll talk more about that very soon…

Has Flashlight helped you? We'd like to hear about it at FlashlightStory@johnhaney.com

In the meantime, you will see a message that you have already completed the puzzle, and then see results with a date of November 30, 2 (yes, the year is 2). This particular side effect is one I will be fixing in the next update of CryptoQuote. Sorry for any confusion this has caused.

UPDATE (Friday, August 5, 2011)

My ticket was assigned within a few hours of being submitted. Since then (now 2 days), I have not seen any other activity. I continue to update the ticket, but am working on making changes to CryptoQuote to work around this problem entirely. I am very disappointed with my web hosting provider and am expecting that my workaround will have to be the solution.

Bottom line, this means the daily challenge will likely be down until you update to a new version of CryptoQuote. A new version of CryptoQuote will take me at least 1 week to get onto the App Store. I thank you for your patience.

UPDATE (Tuesday, August 9, 2011)

My website provider is not willing to fix this issue. Fortunately, I have submitted CryptoQuote 2.1 to the App Store. Hopefully this update will be reviewed and approved within a week.

Thank you all for your patience and kind emails during this time. I am sorry that you are missing out on this regular part of your day and hope to have this back working great for you ASAP. Please know that this new app version will give me the ability to workaround the issue seen this time, as well as provide for a smooth transition to a new web hosting provider should that become necessary in the future.

Thanks again and happy puzzling!

UPDATE (Wednesday, August 10, 2011)

FIXED!!! Apple accepted my request for an expedited review and CryptoQuote 2.1 was reviewed and approved today. You should see the new update in the App Store on your iPhone, iPod touch, or iPad now or within the next few hours. My great appreciation again for your patience and happy puzzling! :-)

I arrived at our holiday house last night in pitch black. The house power was out and I needed to bash through the garden to the electrical meter and back into a pitch black house a couple of times. I discovered to my delight that I had downloaded your flashlight app onto my phone. It saved my life, without it on my iPhone 4 I wouldn't have been able to fix the fuse trip switch or avoid falling in the pond.

Thanks. It's a great app.

Here is the official list of "what's new"…