A journey to simulate and transform consumer experiences using Generative AI in the Home Improvement Sector : challenges & opportunities

?

I am a home improvement enthusiast (lawn & garden is my favorite department)

?

Home Improvement is not often rip and replace, it is also some form of incremental advance in terms of imagination, simulation and creation of new experiences with current consumer investments. (especially in a hyper inflationary macro environment).

?

Today Gen AI can help us create imagery, but cannot help us interact with images, the way we engage with the real world, especially in the Retail, CPG, Travel, and Hospitality sectors .

?

I have seen, successfully implemented many, also failed to realize the value on a few, applications of AI in use cases in the consumer and tech industry space, where I have spent a lot of my professional life.

?

We at STL Digital have built AInnov as our integrated solutioning fabric with AInnov Experience, AInnov Knowledge, AInnov Security and AInnov Digital Twin.

?

We have had good outcomes with our AInnov Knowledge solution, essentially wherever we are working with Textual content (structured, unstructured, PDFs, integrated with SaaS etc).

?

I feel there are lot of problems and opportunities in the multi-modal AI space, which are not realized today in the Consumer industries. On the AInnov Experience Solution front, we have explored on the multi-modal AI space, overlapping with Digital Twins as well.

?

Around October 2022, ?when ChatGPT was? yet to be launched, I had written about ?“The Top 10 opportunities for transforming consumer experiences in home & hybrid workspaces “. ?I was excited to use DALL.E 2, which was released in early 2022 for text2image and image2image interactions to highlight the possibilities.? Then ChatGPT arrived and the whole world of possibilities unfolded at a pace not seen before.?

?

At STL Digital, we have been driving the product management and engineering around an exciting initiative we called as AInnov Experience. I like to call it as? engaging with “Product Catalog”? to engaging with “Experience Catalog”.

?

We filed the patent for fine tuning the interaction between images and scenery based on the initial work on the solution we built using Gen AI.?

?

Today as a consumer, if you go to any retailer’s mobile app for omni-channel experience, you will get different degrees of sophistication in terms of experience simulation and visualization capabilities. The home improvement retailers have a lot of opportunities in terms of transforming experience both for DIY as well as the “Pro” market.

?

亚马逊 , 家得宝 , Ace Hardware Corporation , Menards , 塔吉特百货 , Tractor Supply Company , 宣伟 , Lowe's Companies, Inc. , Floor & Decor ,?Google Cloud , NVIDIA AI , Meta , Unity

#GenAI #RetailTech #CustomerExperience #HomeImprovement #HomeImprovement #Innovation #AR #VR #Metaverse

?

You can see in the images when I tried to simulate a new experience using generative AI, for my challenged:)-veggie garden. It is a broken process, pretty much in anything you want to do on home improvement. You get inspired from multiple sources (social, web, retailer portals DIY ideas, your own network etc.) and then you must manually decide how to bring it all together.?

Pandemic accelerated the engagement, interests and spending on consumer projects. Now with rationalization in consumer spending, there is a need for targeted demand gen based on simulation for all stakeholders in the home improvement sector. There are lot of opportunities for both consumer and pro sector to interact with experience simulations, not just from inspiration perspective but with an integrated “all in one” view.

?

E.g. A Service Desk Order is still complex from a Retail Consumer experience perspective –changes to order on any attribute, service scope, service dates of fulfillment (supply chain visibility), service materials list - stock guarantee on specific parts of the order, design plans, agreement on design implementation, returns. Beyond Home Improvement retailers, if you were to engage with the supercenter retailers on their merchandise assortment for stitching together an experience with a combination of products from the ecommerce catalog of one or multiple retailers, imagine how incredibly hard it is.

?

Gen AI has done well on text summarization, text creation, insights generation, and definitely hallucinations?

?

However, in terms of multi-modal AI especially with digital + physical combination of Gen AI, pre-Gen AI techniques and physical modeling, there is a lot of work to do.

?

I am sharing with product management, experience orgs and technology innovation teams at retailers, travel & hospitality, partners, and ecosystem for driving more growth in this area to transform consumer experiences.

?

Text to Visual imagery creation: Because of Gen AI, AR/VR evolving maturity, now we all have fantastic possibilities especially what we can do in terms of simulation of inspirations from multiple sources.?

?

DALL.E 3, Midjourney, Stable Diffusion do a decent job with the creation of images from textual description. But they lack the capability for direct interaction between text and images , for iteration and continue engagement with additional infusion of real world product images. This means that while Consumers can input text to generate images, there isn't a mechanism for modifying or refining those images through further textual input.?

?

We integrated Meta Quest 2/Quest 3. Apple Vision Pro is plan in progress. The entire AR/VR + Mobile App + Gen AI + Product Catalog + Heterogeneous inspirations (multi-modal) is a really tough one and product management drives the prioritization on where we go on this topic. The AR/VR + Product catalog definitely requires a lot of work for each individual retailer and there is no one size fits all.?

?

We worked on this opportunity from both AR/VR perspective as well as the 2D perspective focusing on how we can augment the current mobile based experience simulation and planning for retailers. ?We have evaluated several existing libraries, used specific open source libraries in conjunction with new approaches, created new techniques, ?evaluated and used multiple Gen AI & data platforms especially Google Generative AI & Cloud services ?, NVIDIA GPU computing, Meta Quest devices & software, UNITY Enterprise, and Vector data platforms, and all major txt2img and img2img libraries on open source.

It is incredibly hard to create simulations of consumer grade experiences with real world data (including but not limited to Consumer's environment and Retailer's Product Catalog) and performing multi-turn iterations of engaging with the simulations through multi-modal input (text , image, audio, video). We have solved few parts of the problem using Generative AI and the extended experience technologies, there are clear opportunities to fine tune further to transform consumer experiences.

?

Few representative challenges in interaction with home improvement experiences - scene simulation:

Orientation:

Objects lose their real-world orientation, affecting the harmony of elements like chairs and tables, any objects engaged and pulled by Gen AI from a Product catalog.

?

Dimensionality:

Objects in generated images do not match expected dimensions relative to the environment.

Research around depth sensors, AR/VR technologies, and taking environment dimensions as input.

?

Visualization of Product:

Inpainting results in blurry pixels, and object size in generated images does not proportionally match the original.

?

3D View in 2D Image:

Challenges in accurately representing 3D views in 2D images, including correct pixel mapping.

?

Experience Iterative Improvement - Model Fine-Tuning Challenges:

Continuous adjustment and experimentation with model configurations demonstrated through detailed logging of the fine-tuning process. Dependency Incompatibility: Issues with software compatibility, especially while stitching together an ecosystem based innovation linked to components, affecting the fine-tuning process.

?

Generated Image Quality:

Enhanced image quality and realism in generated images after fine-tuning is an area requiring huge ecosystem progress.

?

Fusion of New Product introductions based on prompts on existing image backgrounds:

?

Unifying and making the new image inclusive of new products while preserving the resolution and integrity of the existing consumer environment

?

?

Experience Attribute planning linked to cost of the underlying product catalog:

Both for the DIY and Pro segment, I see great opportunities to apply a composite Gen AI approach to transform experiences while allowing consumers to iteratively plan and engage with products and develop experience options linked to cost.

?

Please share your thoughts on this topic. My Team and I will be happy to demo , discuss, and engage on this usecase and related technology innovations. Please write to me or contact us at [email protected]

?