Making Data Less Dense

BrightAI is an enterprise AI + IoT (AIoT) solution platform for accelerating digital transformation of the physical world.

MY ROLE

I was the design lead, responsible for the end-to-end design process of the BrightAI platform, ML Development Studio, and Spark, the BrightAI design system

TEAM

Sam Carlton, Staff Front-End Engineer
David Rudman, Senior Product Manager
Prakarn Nisarat, UX Designer
Mark Jaremko, VP of Product
Nancy Li, VP of Platform
Haidong Wang, VP of Engineering
Ben Seefeldt, Senior Mobile Engineer
Brad Marsh, Senior Engineering Manager
Luke Bredeson, Staff Engineer

VERSION

v1.0

THE PROBLEM

It’s hard for traditional offline businesses to offer intelligent products and services to their customers. How can we easily and efficiently bring these physical environments online, and apply AI to IoT to make better use of data?

GOALS

Defining Data

The goal was to build an end-to-end platform solution that allows businesses to easily and efficiently adapt AIoT to monitor their physical spaces with greater understanding and actionable insights.

High level goals:

• Simplify the complex by creating an easy to use experience and user interface
• Create a modular solution that will scale for various use cases
• Create a machine learning feature that will allow users to easily build, train, and deploy AI models

Hypothesis:

• We believe that digitizing physical spaces for admins managing/monitoring fleets of devices will enable companies to avoid unplanned downtime, increase operating efficiency, and enhance risk management

SOLUTION

Simplifying the Software

The BrightAI platform is an end-to-end low-code AIoT solution development platform that makes it easy for any business to train machine learning models in order to sense and interact with their existing physical spaces, machines, workforce and consumers. The platform makes it quick and easy to identify, build, train, deploy and improve edge, and cloud AI models at scale.

DISCOVERY

Learning Machine Learning

While I had extensive experience working with data, AI was new territory for me. As a first step, I developed my knowledge base and expertise: I studied competitors; attended demos and tutorials; and learned how other products implemented machine learning tools, event monitoring, device management, and effective use of data. Additionally, I had extensive conversations with key stakeholders and the engineering team about the vision for the Bright platform.

Competitors include:

• Particle.io
• C3.ai
• Instabase
• Google Vertex AI
• Microsoft Azure
• Datadog

Key insights:

• There is no current AIoT solution
• Machine learning is currently only used by a small group of expert data scientists and machine learning engineers
• There is a big data problem in AI and IoT. Data needs to be used more effectively
• As a platform, we need to support industry specific verticals at scale
• There are many varied users and use cases

1. Google Vertex AI; 2. C3.ai; 3. Datadog; 4. Particle.io

"The value of AI in the context of IoT is its ability to quickly wring insights from data. Machine learning brings the ability to automatically identify patterns and detect anomalies in the data that smart sensors and devices generate—information such as temperature, pressure, humidity, air quality, vibration, and sound. Compared to traditional business intelligence tools—machine learning approaches can make operational predictions up to 20x earlier and with greater accuracy."

Challenges

Current user challenges:

• Large field service workforce with limited digital tools, results in reactive service, high labor churns, and high cost of failure.
• Large legacy fleet of physical machines and spaces, with limited data for actual service cost and operating efficiency.
• New digital challengers redefined consumer expectations for personalized digital experiences, causing traditional businesses to suffer customer churns and revenue loss.
• Lack of in-house tech and business transformation expertise for AI + IoT make it hard to move quickly to scaled deployments.

From here I began formulating questions:

What does the internal team (solution engineers) need to support the fleet and what does the customer need to support their fleet?

At a high level, users need visibility into the fleet and control; a console and admin portal with access-based control/permissions; a reference application so customers can build their own portal, catering to engineers of organizations we are servicing; mobile apps, libraries and SDKs that will be reused for each customer.

The solution would need to support the following:

• BrightAI solution engineer – Bright Studio (including ML Development Studio)
• Tenant admin – Bright Studio/customer portal (including the Link App for field technicians)

    BrightAI admin use cases

    Tenant admin use cases

IDEATE

How Do We Make the Old New Again?

To kickstart the ideation process, I redefined the problem statement:

How might we enable companies to avoid unplanned downtime, increase operating efficiency, create new products and services, and enhance risk management?

Design Principles

Working with stakeholders and product, we developed the following platform principles:

Scalability
Developers and admins should be able to easily set up customer solutions

Flexibility
Core platform components need to accomodate all solutions while allowing for customization

Reliability
Critical functionality operates on the edge

Efficiency & Ease
All platform components are composable and installation is frictionless to ensure rapid deployment

Bright Studio is an umbrella term to describe multiple apps (customer solutions, admin portal, ML Dev Studio, among others); a shared app shell application model with common components and conventions across apps. The Bright design system ensures consistency across apps and solutions.

When building a platform product, scalability is of the utmost importance. Keeping this in mind, I established three templates:

1. Dashboard/Overview Page
2. Data Page
3. Details Page

Dashboard / Overview page

Data table page

Details page

Fleet Management

Exploring the fleet management use case, I identified Devices, Locations, and Events as the most crucial elements for monitoring an IoT fleet.

As an admin, I would like to have a snapshot view of all the information important to monitor the health of our fleet, in order to quickly identify issues as they happen.

     Fleet Management onboarding a device

Working closely with the stakeholders, product, and engineering, I began to establish the architecture for the platform. Once the core modules were established, I began exploring whether we choose to show multiple solutions (apps) enabled for the customer in one aggregated view, with the option to filter to a solution, or have a separate entry point to each solution.

(Left) solution-focused information architecture; (right) tenant-focused information architecture

Tenant portal information architecture

Working with assumptions, I began with what I know: creating portal solutions for businesses and consumers. While SaaS and PaaS products are similar, there are differences in complexity and workflows. This was verified by talking to admins and technicians.

Solution-focused fleet management wireframes

Tenant-focused fleet management wireframes

As a customer managing an IoT fleet, I want a solution that is easy to integrate and manage.

Location empty state

Add location

Locations page

Location details page

Add device to location

Room details page

ML Development Studio

Exploring the use case of using AI/ML to predictively analyze IoT, in order to proactively identify issues, the solution was to create the BrightAI ML Studio: a low-code collaborative AI development platform, where developers, with limited ML expertise, can easily augment data, train, deploy and improve edge and cloud AI models at scale.

ML Studio provides repeatable ML workflows for computer vision object detection and image classification models.

There are 5 main steps for ML Studio:

1. Import (Datasets)
2. Label (Images)
3. Train (Model) or Customize / Improve Existing Model
4. Evaluate 
5. Export & Deploy

Create model experiment flow

Initial Label and Train wireframes

Datasets page

1. Import Datasets
The BrightAI model builder imports videos and images, so that BrightAI’s autoML will automatically analyze and pre-label images and videos, based on pre-trained models like people/machine/activity classification, object detection etc.

Label data object page

2. Label Images
The BrightAI model builder is then able to review the pre-labeled tags on all uploaded images, and correct any of the frames that are tagged wrong or not tagged, so the model can be properly trained.

Data Labeling Journey

3. Train Model

• Basic model training
  • A ML non-expert can select their target hardware and click the train button. Training happens automatically. BrightAI studio auto-selects model configurations based on image types and target hardware.

• Advanced model training
  • An ML expert can create a set of model experiments and set up different model configurations to optimize for the best possible model, balancing precision, recall, device memory, prediction latency, etc.

Training Journey

4. Evaluate
The model builder then views the Experiment details and sees the key metrics for evaluating the model.

Evaluate results

5. Export & Deploy

• Export & Test
  • The new model is tested on a specific device (camera) to see if it truly performs well. The test camera feed can be streamed into BrightAI Studio, where real-time prediction results can be viewed.
    

• Export & Scale Deploy
  • The model is exported and integrated to the AIoT endpoint software and a cohort of devices is select and deploy.
    

Deploy

REFINE

The Final Solution

The BrightAI platform makes it quick and easy to identify, build, train, deploy and improve edge and cloud AI models at scale. Business can easily train machine learning models in order to sense and interact with their existing physical spaces, machines, workforce, and consumers with a simple, low code solution.

Fleet Management Overview page

Fleet Management Locations page (map view)

Fleet Management Devices page (filter view)

Fleet Management Device Overview page

     Locations page (default view)

     Locations page (map view)

     Device Overview page

Fleet Management Overview page (dark mode)

BrightAI Studio simplifies the process of machine learning and AIoT integrations into four easy steps.

With a focus on research, best practices, and a team of subject matter experts, we were able to build a product that is scalable, reliable, flexible, efficient and easy.

The design of BrightAI Studio is contemporary, and optimized for complex data structures, and consistency of components and patterns with the implementation of Spark, the BrightAI design system.

Data table with surfaced filters

Data table with hide/show filters

Trade Offs

The Bright platform is an extremely data-dense product. To keep the UI clean and clutter-free, and maximize screen real estate, some content is not surfaced. However, since this is an enterprise product, most users will be using the product often, and will require some additional discoverability as a new user. Given these trade-offs, the team decided they were worthwhile, allowing users to focus on tasks, and promoting consistent pattern usage throughout, creating a better user experience.

Design Challenges

Challenges included working with a highly complex product and architecture; not-so-tech-savvy customers; building a design system in tandem; solving for single tenant vs multi-tenant solutions, and on-demand cloud connectivity.

Learnings

Like most enterprise applications, it is necessary to support key workflows and support the existing business processes. However, with BrightAI Studio, we are seeking to change existing processes and workflows. We are not simply helping customers do their job efficiently. We want to help them do their job better.

People are averse to change. Making archaic business models modern is challenging both technically and behaviorally. You can’t run there. You must crawl first, then walk, and then run.

IMPACT

Results

As we move toward improving BrightAI Studio, next steps include, expanding ML Studio to support predictive data and other model types; dark mode support; designing and build additional modules; design and implement a multi-platform design system.

• February 2022 we released v1 of the design system
• August 2022 we released the alpha version of Bright Studio
• We continually improve our AI predictions with an over 80% accuracy rate
• Alpha solution was launched in < 9 months