Generating Pixel Art from any uploaded image with Tensorflow.js

And lots of vanilla Javascript...

Hi! I'm Tristan Bains, a Dutch data scientist from the Amsterdam area in the Netherlands.

I wanted to add a fun, end-to-end project to my data science portfolio. I also wanted to use TensorFlow.js, the JavaScript version of TensorFlow, which I hadn't used before. While I had previous experience with TensorFlow's Python version, I was new to its JavaScript counterpart.

I considered many ideas but ultimately chose to create a project that converts user-uploaded images into pixel art. To build this quickly, I decided to use tools I already knew: Python Flask and TailwindCSS. This way, the only new thing I had to learn was TensorFlow.js.

Looking back, while I gained valuable experience working with TensorFlow.js, I spent most of my time designing a user-friendly experience. You can read more about my reflections at the bottom of the page.

Table of contents

Upfront requirements

Before starting, I wanted to ensure that the project met a few key requirements:

User Interaction and Code Flow

TOnce you load the homepage, here's what happens:

  1. Image Upload: You upload an image of your choice.
  2. Pixelation: A JavaScript listener activates TensorFlow.js, which creates multiple pixelated versions of your image. These pixelated images are stored as SVG strings in your browser's memory.
  3. Display and Variation: The pixelated images are displayed as different image formats, such as Lego-style, circular, and square grid.
  4. User Controls: Once the initial images are displayed, you can adjust them using various controls. The user controls become visible at the bottom of the page
  5. Further Interaction: You can continue to interact with the website by uploading new images, editing existing ones, or downloading the generated images.

Tech stack

Front-end

Back-end

While the website appears dynamic, it's actually a static web application, eliminating the need for a traditional database.

Other

Reflections on the project

The deep learning aspect

Initially, I was excited to dive into deep learning with TensorFlow.js. However, the core deep learning component ended up being quite straightforward. It primarily involved slicing the uploaded image and applying a simple 2D convolutional layer. I spent more time optimizing the browser's memory management for tensor operations than on the actual model architecture.

User experience

The most time-consuming part of the project was refining the user experience. Early versions were confusing, requiring numerous info boxes to guide users.

To improve this, I adopted a minimalist approach, removing unnecessary options and streamlining the workflow. I also implemented lots of JavaScript listeners to make user interactions more intuitive and responsive. Because of this, I picked up a lot of JavaScript.

Front-end

A few months ago, I learned about DaisyUI. I liked it so much that I decided to use it for this project. It made building the website even faster than using just TailwindCSS.

I hope you like the project. Feel free to contact me anytime.

Tristan

Designed and built by Tristan Bains in Amsterdam, NL