Elevating the world’s first open-source design platform with AI

Penpot partnered with Neurons Lab to transform creative and design-based capabilities for their users by leveraging generative AI technologies.

Partner Overview

Penpot, with a growing user base of +600k, is the first open-source collaborative design and code tool.

Their mission is to provide an open-source and open-standards platform to take collaboration between designers and developers to a new level. Their aim is to be the ultimate ally for designers, enabling them to achieve unprecedented scalability and creativity.

Having scored $20m in funding last year, Penpot partnered with Neurons Lab to develop new design possibilities for their users through generative AI technologies. This ongoing project began with a deep-dive R&D approach and has evolved into the integration of Penpot’s first AI-based design feature.

Project Objectives 

Penpot had some key objectives in mind for their AI project:

Firstly, they wanted to empower designers to create in a more efficient way so they could dedicate more time to creative tasks. 

Secondly, being an open-source platform, they wanted to strengthen and bring more value to their community by building community-powered and owned design systems. 

Thirdly, they wanted to integrate AI into the product in order to stay competitive. Some of the features they looked to develop were first of their kind and would give them a serious competitive edge if successful. 

And when Penpot got in touch with Neurons Lab, they already had a series of AI features in mind that they wanted to integrate into their design platform.

After assessing feasibility and potential impact on users and business, two features were selected to work on first:

1. AI-Powered Design Co-Pilot

Penpot wanted to build an innovative Design Co-Pilot powered by AI which would allow users to describe desired changes in a text format, leaving the heavy lifting of SVG element editing to the AI system. The Co-Pilot would streamline the editing process, eliminating the need for manual adjustments.

2. UX2Doc feature 

Penpot also wanted to create a UX2Doc feature to transform design ideas into comprehensive documents using generative AI. This feature would automatically generate technical and functional UI/UX specifications, freeing up time for designers to focus on creative tasks.

Challenges

Although Penpot had already identified the AI features they wanted to work on, they faced a few challenges along the way. 

Firstly, Penpot could not just buy an “off-the-shelf” solution because they wanted to create an open-source solution that would be available to their community. Furthermore, the necessary open-source datasets don’t currently exist online, which means there was a limited dataset for training, the data would need to be created synthetically. 

In addition, at the start of the project, Penpot’s tech team did not possess the necessary expertise to develop the kind of features they aspired to build. 

Solution Overview

During the ongoing partnership between Penpot and Neurons Lab, fundamental progress has been made in order to research and integrate the two priority AI-based features: AI-Powered Design Co-Pilot and UX2Doc feature. 

Before beginning development on specific features, Neurons Lab created an MLOps pipeline and internal toolset for all of Penpot’s future AI development. This provided an experimentation framework where Penpot could quickly run different experiments and develop new AI solutions.

AI-Powered Design Co-Pilot

Since there is no existing technology that could meet Penpot’s requirements for the design co-pilot, the solution needed to be developed from the ground up. 

After exploring several options, Neurons Lab chose to leverage state-of-the-art techniques by integrating Diffusion-based Models and Large Language Models (LLM) into the AI Co-Pilot solution.

From refining icons to handling complex SVG manipulations, the combination of creativity and technology empowers users to work faster and optimize design outputs. 

This feature is now an advanced prototype, having created a data feedback loop with customers to refine the model. 

UX2Doc feature

Now at the commercialization phase, Neurons Lab created the UX2Doc feature to transform design ideas into comprehensive documents using generative AI. The feature uses Amazon SageMaker JumpStart and Llama 2 to automatically generate technical and functional UI/UX specifications. 

With the feature, developers can do documentation in real-time and focus more on creative tasks, vastly improving the productivity of both individuals and entire teams. The feature has already been deployed to end users. 

Results 

• Penpot is now able to quickly run different experiments and develop new AI solutions thanks to their new MLOps pipeline
• Penpot has an advanced prototype for its Co-Pilot solution
• Penpot users can now access a fully functioning UX solution which increases the quality and efficiency of documentation

Next steps

• Penpot continues to work on its AI efforts, planning the development of their next features
• Neurons Lab will build a Data Science Center of Excellence for Penpot so they can continue AI solution development in-house
• Neurons Lab are currently gathering datasets, training models, and continuing development on Penpot’s Co-Pilot solution

About Neurons Lab

Neurons Lab is an AI consultancy that provides end-to-end services – from identifying high-impact AI applications to integrating and scaling the technology. We empower companies to capitalize on AI’s capabilities. 

As an AWS Advanced Partner, our global team comprises Data Scientists, Subject Matter Experts, Cloud Specialists supported by an extensive talent pool of 500 experts. This expertise allows us to solve the most complex AI challenges, mobilizing and delivering with outstanding speed and supporting both urgent priorities and strategic long-term needs.

Ready to leverage AI for your business? Get in touch with the team here.