Welcome to Lowpass, a newsletter about the future of entertainment and the next big hardware platforms, including smart TVs, ambient computing and AR / VR. This week: Amazon is building an XR projector, and a former Apple Vision Pro developer launches a new startup.

This week’s Lowpass newsletter is free for all subscribers; next week’s lead story will only go out to paying members. Upgrade now to not miss it.

Amazon’s ambient XR strategy comes into focus

Amazon used its fall hardware event yesterday to introduce a number of new devices, including two new Fire TV sticks, a new Echo Show smart display and a $600 router (nope, that’s not a typo). Company executives also highlighted how Amazon is advancing Alexa’s conversational capabilities with LLMs, and how it is integrating services powered by third-party LLMs into its voice-enabled devices.

Curiously absent was the leftfield hardware that typically gets unveiled during these fall events. There was no home automation robot, no security drone designed to fly around in your house when you’re not at home, and no creepy camera looking to watch you get dressed in the morning.

However, despite layoffs in its hardware division, it appears Amazon isn’t done with pushing the boundaries of new hardware. Reuters reported this week that Amazon is working on a projector capable of “​​turning regular surfaces into screens.” Amazon acquired San Francisco-based projection mapping hardware startup Lightform to kickstart these efforts, according to Reuters. An Amazon spokesperson declined to comment.

That revelation is super interesting. Lightform built what it called an “AR projector,” and Amazon has long explored using projection mapping to create interactive displays on a range of surfaces.

What you need to know about Lightform. San Francisco-based projection mapping startup Lightform had built two hardware products: A depth camera that could turn any projector into a projection mapping device, as well as an all-in-one projector. It also built an authoring solution for projection mapping content.

• The company was targeting museums, retail locations and location-based entertainment venues for its initial hardware products – a market that froze up when COVID hit in early 2020.

• Lightform then tried to pivot to in-home projection mapping, but its all-in-one LF2 projector was simply too expensive for most people. Plus, it didn’t help that some of those projectors literally caught on fire, forcing Lightform to initiate a recall.

• Lightform announced in February of 2022 that it hadn’t been able to secure additional funding, and that it was winding down its business later that year.

• LinkedIn resumes suggest that the company had actually been taken over by Amazon even earlier, with a number of staffers transitioning to Amazon’s Lab126 hardware unit in July of 2021.

• These include Lightform CEO Brett Jones, who is now heading a new product team with a ten-year roadmap, according to his LinkedIn profile.

• Earlier this year, Jones was looking for a design technologist to join his team. Among the preferred qualifications for that job: “Experience designing 3D spatial interactions for AR/VR or games using tools like Unity, Unreal Engine, etc.”

This isn’t the first time Amazon has eyed projection mapping. The company introduced a projection-based video calling device for kids and their caregivers in 2021. The Amazon Glow used projection mapping to turn your tabletop into an interactive screen, but its focus on remote interaction may have been a bit too narrow: Amazon discontinued the device in late 2022.

• I wrote a behind-the-scenes feature about the creation of the Glow last year, and some of the team members who had worked on the project told me at the time that they were looking to bring similar technology to elder care and other use cases. (The Glow team disbanded when the product was killed.)

• Years before that, Amazon had explored the idea of using ceiling-mounted projectors for glasses-free AR experiences.

• The company picked up on some of those ideas last year when it began hiring folks to create a “magical and useful, new-to-world XR consumer product.”

• One of the roles Amazon was looking to fill at the time was that of a UX designer to work on “applications spanning from multi-modal interfaces to 3D AR entertainment experiences.”

• Applicants for that position were told they would have to be able to “think spatially, with 3D design experience in motion design, animation [and] AR/VR, games.”

Amazon’s upcoming projector will need to know your house. Reuters reported this week that the company’s in-development projector will allow users to “beam recipes on the wall above their stove or make Zoom calls that track them as they move.”

• That will require a lot of spatial awareness. Interestingly, Amazon launched a new “Map View” feature for its Alexa app this week that allows people to scan their surroundings with their iPhone, and then create a 3D map to visualize smart home devices throughout their home.

• Map View appears to use the iOS RoomPlan API to create a 3D map of every room in your house, including furniture and other surfaces – a level of detail that’s overkill if you just want to turn off the lights room by room, but that may come in very handy if you wanted to use all those surfaces for other purposes in the future.

• Amazon also unveiled a number of other features this week that add new interaction modalities to ambient computing. The new Echo Show 8 smart display adjusts what it shows to users based on their proximity, with an option to further personalize content for people who opt into facial detection. And the company is adding eye gaze control to its Fire Max 11 tablet, further hinting at a future in which we don’t even have to talk to Alexa anymore to control it.

Projection mapping is hard. The technology makes for great demos and flashy art projects, but getting it to work in everyday environments can be challenging. People’s homes are vastly different, and using those homes as interactive screens is way more difficult that just using a screen you control.

At the same time, there’s huge potential here, even beyond the scenarios cited by Reuters. Amazon’s Glow device hinted at some of those opportunities by letting kids scan their own objects that could then be incorporated into stories and games.

One could imagine projectors that work on much larger surfaces, and perhaps even 3D environments, to create mixed reality experiences in your home. What if a projector could let you play games, with objects bouncing off your walls and furniture, all without you having to put on AR glasses or a VR headset? Or what if you were in an AirBnB, and you could Alexa to point out the cupboard with the wine glasses with a targeted projection?

Granted, we won’t know if Amazon’s take on projection mapping will enable any of this until the company launches it. But I, for one, can’t wait for the company’s next hardware event with wacky, leftfield products.

One to watch: Volumetrics

Former Apple Vision Pro software engineer Michael Gutensohn has launched a new company called Volumetrics that aims to enable mixed reality developers to build their apps without ever having to take off the headset. Gutensohn is being joined by Laurent Baumann, who also worked at Apple a few years back, albeit not on the headset.

Volumetrics is announcing Thursday that they have raised $1.1 million in seed funding, and Gutensohn told me this week that his goal is to build a company dedicated to openness. The duo is using Mastodon as its Slack alternative, meaning that anyone can follow their development process, and is betting on WebXR as a way to make sure that its platform will run on a wide range of current and future devices.

That being said, Gutensohn was clear-eyed about which comopany is dominating the space for the time being: “Our first corporate purchase is going to be some Quest 3s,” he told me.

What else

Microsoft just leaked all the things. In its legal spat with the FTC, Microsoft accidentally disclosed its multi-year roadmap for upcoming Xbox products. Yikes!

The Pirate Bay turns 20. Two raids, a bunch of lawsuits and countless take-down notices couldn’t put an end to The Pirate Bay … but arguably, streaming has made it a lot less relevant.

TikTok is introducing labels for AI-generated content. The service is imploring creators to label any AI-generated, realistic images, audio or video as such — but will they listen?

Sonos scores another point in its fight against Google. Google’s attempt to bring the issue back to the ITC appears to have failed.

Longtime Microsoft exec Panos Panay is heading to Amazon. Panay will reportedly replace Dave Limp, who has been in charge of the company’s consumer hardware products.

Varjo cuts price of its PC VR headset by 50%. That’s probably not a good sign?

That Chinese live streamer may be an AI. Super interesting Technology Review story about Chinese live shopping streams increasingly relying on AI-generated hosts that can hawk their wares 24/7.

That’s it

This is fun: The New York Times chronicled the evolution of tech based on Sandra Bullock movies.

Thanks for reading, have a great weekend!

Photo by Steven Weeks on Unsplash