Computer Vision methods provide great flexibility for images processing. For example, you can easily write a computer program for converting any photo to an artistic sketch. In this article I will show you:
- how to create Python code for converting images into sketches in Jupyter Notebook with the usage of OpenCV package,
- how to convert the notebook into a web application with Mercury framework,
- how to deploy the web app to Heroku (using free dyno).
The preview of the application is presented in the GIF below:
Looks interesting? Let’s start - you will learn a lot! 🤓 📚
Setup a GitHub repository
Please create a new repo for our project. You can do this by going to https://github.com/new. I’ve selected
artistic-sketches-jupyter-mercury as the repository name. After repository creation, please click on a green
Code button and copy the address to clone the repository.
git clone email@example.com:pplonski/artistic-sketches-jupyter-mercury.git
My repository is public and available on GitHub https://github.com/pplonski/artistic-sketches-jupyter-mercury.
Set virtual environment and install requirements
Let’s change the directory to the project folder. We will use
virtualenv to create a virtual environment for our python packages.
virtualenv sketchenv source sketchenv/bin/activate
sketchenv is the name of our virtual environment. Let’s create
requirements.txt file with needed packages:
mljar-mercury opencv-python-headless pillow matplotlib
Please install packages by running:
pip install -r requirements.txt
To use the new virtual environment in the Jupyter run:
python -m ipykernel install --user --name=sketchenv
It is important to add a virtual environment as a kernel in Jupyter because it is not the same thing as the Jupyter kernel.
Create a new Notebook
Please start Jupyter Notebook with the following command:
Let’s create a new notebook. ** Remember to select a kernel with the name of the virtual environment
sketchenv.** Change notebook’s name by clicking the
Untitled at the top. I set
sketch_app as a new notebook name.
Start with importing packages and setting the
matplotlib plot options.
import os import cv2 from PIL import Image from matplotlib import pyplot as plt import matplotlib as mpl mpl.rcParams['figure.figsize'] = (12,12) mpl.rcParams['axes.grid'] = False
Let’s set new variables in the next cell:
image_path = "/home/piotr/Downloads/house.jpg" output_dir = "output_images"
image_path is a path for an image that will be converted. The
output_dir is the directory name where a new image with sketch will be saved.
In the next cell I will define a method for image plotting in the notebook:
def imshow(image, title=None, fname=None): if len(image.shape) > 3: image = tf.squeeze(image, axis=0) plt.imshow(image) plt.axis('off') if title: plt.title(title) if fname: plt.savefig(fname)
Let’s load the image and plot it:
img = cv2.imread(image_path) RGB_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) imshow(RGB_img)
You should see the house like in the image below:
Now let’s apply the sequence of image transformation that will produce the artistic sketch of the image:
# image transformation grayed = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) inverted = cv2.bitwise_not(grayed) blurred = cv2.GaussianBlur(inverted, (19, 19), sigmaX=0, sigmaY=0) def blend(x, y): return cv2.divide(x, 255 - y, scale=256) final_result = blend(grayed, blurred) # save the image _ = cv2.imwrite(os.path.join(output_dir, "sketch_photo.jpg"), final_result) # plot the image RGB_img = cv2.cvtColor(final_result, cv2.COLOR_BGR2RGB) imshow(RGB_img)
You can read about step-by-step image preprocessing on Behic Guven’s article.
The image after transformation should look like this:
OK, you have the notebook ready, but how to share it with others?
Mercury for notebook sharing
The Mercury framework will help you with notebook sharing:
- it transforms a notebook into a web application,
- sharing is as easy as sending the link,
- the input widgets for the notebook are created based on the YAML header,
- user can provide her input and then execute the notebook,
- the notebook’s code can be hidden - to not scary non-coders.
The Mercury is already installed in the virtual environment (the
mljar-mercury package). Let’s start the web app from the notebook by running:
mercury watch sketch_app.ipynb
Please open the web browser and enter the address http://127.0.0.1:8000/. You should see your notebook’s card:
Please click on the notebook, and you will have it displayed with the warning message telling that YAML header is missing:
Let’s add the YAML header. Please add the RAW cell at the top of the notebook.
The YAML header:
--- title: Convert Photo to Sketch 🖌️ description: Create sketch from uploaded a photo show-code: False params: image_path: input: file label: Please upload a photo output_dir: output: dir ---
The YAML header configuration:
title- the application title used in the home view and at the top of the sidebar in notebook view,
description- the text describing what the app is doing, it is used in the home view,
show-code- the variable which decides whether to show or hide the notebook’s code,
params- the widgets added to the notebook.
There are two widgets added to the notebook:
filewidget. The variable
image_pathwill be replaced with the path to the uploaded file.
output_dir- the variable pointing to the output directory. All files created in the output directory will be available for download.
The notebook has now the file upload widget, and here is what it looks like:
Deploy to Heroku
Please set Heroku account and install Heroku CLI tools. You can create the Heroku app by running:
heroku create sketch-app-mercury
To make the application work on Heroku, we will need to add two more files. The
Procfile with the command how to run our web app:
web: mercury run 0.0.0.0:$PORT
Aptfile with additional system libraries that needs to be installed in order to use
libsm6 libxrender1 libfontconfig1 libice6
To install additional system libraries, we need to add buildpack in Heroku:
Please open your Heroku dashboard and go to app settings. There please find the
Buildpacks and add the above link.
Please add and commit all project files to the repository. The application can be deployed to Heroku with the command:
git push heroku main
My app is running at https://sketch-app-mercury.herokuapp.com/.
Thanks to tools like Python, OpenCV, and Jupyter you can easily create amazing scripts for images manipulation. The open-source framework - Mercury is the easiest way to share the notebook with non-coders. You can easily add input widgets to the notebook. What is more, the code can be hidden, and only outputs are displayed to the end-user. The deployment process is simple. The Mercury is built on top of the Django framework. Thanks to this, it can be deployed on any server in the cloud.
- The sketch app repo 👉 https://github.com/pplonski/artistic-sketches-jupyter-mercury
- The sketch app running at Heroku 👉 https://sketch-app-mercury.herokuapp.com/
- Mercury GitHub repo 👉 https://github.com/mljar/mercury