Would you share your indoors for the rest of the world to see?
Cartographers are mapping where 80% of our time is spent
It was a hot summer afternoon on the USC campus and the session on “Mapping the World in 3D” was underway in a classroom filled with resolute engineers and “data enthusiasts.” This one proposed a shift in human experience in an otherwise watered down series of sessions at Big Data Day LA 2018.
With an engineer’s passion, Ryan Measel, CTO of Fantasmo, held little back. He spoke with a calm monotone in a disturbingly-honest style. It’s a manner of open talk I respect when done with compassion: big vision and idealism.
I felt myself playing campaign manager like I was back in politics; each statement was an unknown risk. My hands clenched and unclenched, waiting for the ‘candidate’ to say something shocking.
I’ve noticed this style of communication in Engineering managers and scientists, who filled a good portion of the seats at the session. When you address people who talk mostly to machines — and they ask questions about how your machines talk to other machines — the ethics and human elements are often left behind as everybody goes technical.
Ryan would go on to specifically cite the ethical and legal problem, hinting at the moral problem, of invading and mapping indoor spaces. Giving permission and scaling storage via the blockchain. As more companies take on the challenge, the solution to privacy and “opting out” seems unclear in the rush to compete and establish AR and mapping standards.
Fantasmo is a handful of Ph.D.’s looking to take on the world — or at least map all the indoor space on it. Here’s a primer from their site:
Immediate thoughts ranged from product strategy and UX to the philosophical:
- Is it realistic to expect mass capture from a community of public cartographers?
- Is there really enough decentralized storage for all this data?
- Is this some kind of reverse VR project where we capture, update, and modify our spaces and experience from the designs of the real world?
My field notes followed his pitch, which answered the big, simple questions for Fantasmo:
Why?
- Spatial applications need to know: “Where am I, What is around me?” “Real-world” autonomous interaction requires it.
- The cost of “reality capture” tech is decreasing rapidly (sensors).
- They get paid. Indoor mapping services are in demand. Like many software startups, the services revenue and customers let them test and build out their platform.
What Opportunity?
- Adding another dimension to maps, the mapping and hosting is expensive. Ex: 3D construction for a mall is 20GB.
- Ex: Uber spending $500MM to improve US maps alone
- A dynamic world, constant movement and change, transient human/objects.
- “Semi-permanent” objects :Ex: conference room chairs. They will stay for years, but move all the time.
- Outputs depend on use: wireframes and basic structures may be all a cleaning bot needs. Blueprints and accuracy matter for … bigger operations (?)
“Social /Legal implications”
- The same laws currently applied to photography [should?] apply to mapping
- “reasonable expectation of privacy” — indoor space is seen differently, enclosed spaces are [more private] property
- Only pure solution is having the first party host the data. You own and host data (i.e. a homeowner chooses who sees inside)
How?
- Lots of 3D data → Distributed storage. Network design — push the data to the edge, have the data representing the building stored in the building or nearby, only accessed when needed, when nearby or onsite
- Location DNS — resolve a GPS query for data, authenticate based on location verification
- Registered geospatial boundaries (i.e. property lines)
- Owners set resolvers and access permissions
- Settings stored in blockchain, where owners set rules
More on the Tech?
- Once authenticated, can pull down maps from distributed storage, which enforces access control
- Stored as binary blobs. with octrees to sort/replace/merge
- Standards-interoperable data sets (creating indoor mapping data standards)
- CPS (Camera Positioning Standard) — relative positions compared to camera angle, positions sent to requesting machine. Use geometry to set accurate location from image (!)
- Spatial Versioning Format: 3D Semantic JSON — marking up 3D data is difficult, filtering is necessary to receive only needed points
- Network effect of sensors, share one base map and update it among themselves, continuously share
- Tooling — browser based rendering for any laptop, 30GB maps
As the session closed, a line of engineers formed to go even further into the tech stack with Ryan. I finished off my notes and walked back to central campus. A final statement on the current duplicative efforts swam through my head:
50 different companies are mapping and remapping San Francisco streets. Leaders will emerge to spread the best techniques to expand to the rest of the world.
Where is all this captured data going?
Fantasmo can’t contain it all on their learning experiment of a blockchain for indoor property maps. Fantasmo’s CEO himself notes the industry is “creating a fractured landscapes of walled data gardens.”
A Camera Positioning Standard is important for making images “interoperable” among machines. It creates shareable data stores. Moving fast and breaking privacy seems likely.
Assassination attempts and surveillance concerns aside, how are we deciding who owns and sees what? I see an internet-famous moment leading to the mob effect with objects and people of interest.
What might trolls do as they watch indoor mapping feeds?
What happens when one of these startups gets acquired?
Google Maps’ Street View was infrequent enough to only have a set of randomly captured people of interest. It felt powerful, omniscient, and a little scary to be able to find your car near your house, where it was parked a few weeks ago on a random Saturday. Jumping down the street you grew up on inspired music video projects customized to your childhood via Google Maps.
Google’s Visual Positioning Standard for where to place cameras could easily be extended and optimized for the indoors — if they were to acquire someone like Fantasmo.
[Google Home] “Your Nest camera should be five centimeters below the ceiling for a complete indoor map. Your map will be available under My Maps in your Google account within a few days and will be updated daily.”
Do photography laws still apply to digital twins?
If technology allows for a re-creation without taking a direct picture — whose property is that? Who gives and needs to get permission? Could an innocent selfie in a bedroom generate enough reference points for a blueprint?
With the “best techniques” spreading — what is the intention, best at what?
Real-time, continuously-tracked objects in any space, including people — there’s almost a cliche of disturbing and dystopian lessons from “being watched” by Big Brother, Google, or the NSA.
We can already quickly piece together clues from metadata on shared pictures to determine exact locations. When Fantasmo’s Ryan spoke of triangulating positions from a single camera source or image capture, my imagination went straight to actually seeing a digital recreation of your indoor space. Your identity and gym shorts mapped and updated as you move them and as you move.
Jane Jetson might have needed Rosie the maid to be updated on the new interior decorations.
Cleaning robots might need to know where a table has moved in a suburban home. Finding lost things and even children is an incredible potential.
Dystopia to balance:
