PyCon Ireland 2019 – Dash: Interactive Data Visualization Web Apps with no Javascript – Dom Weldon

>>All right everyone thank you for being
here this morning, we start with the first session this Sunday with Dom who will tell
us about visualisation with Dash. MR WELDON: Hello, great, thank you everyone
for coming, this is the conference hardcore, the first session of the second day. But as I said thank you, my talk is Dash,
or interactive data visualisation web apps without using Javascript. So an alternative title for this talk would
be what you can, can’t, should and probably shouldn’t do with Dash, which is a project
by Plotly. So for the remainder of of this talk I’m going
to go over a couple of areas, first off explore why you might want to
use Dash, then give a brief introduction to how Dash works and what kind of things you
can do with it, and then a couple of the lessons we’ve learned from my company, Decision Lab,
about things you might want to think about as a Dash App grows or you incorporate Dash
into your team. But please do shout out with any questions
if you have them or feel free to approach me later on if there is an area that particularly
interests you. So just before we go on I should offer a brief
disclaimer, I am not an expert necessarily on Dash, this talk is really about using Dash
at work, I’m not an author on the project, I hope one day I will bump into someone who
is an author on the project, so if anything is incorrect or a little out of date please
let me know. There is another question really, which is
why am I here in the first place? More to the point I’m full stack developer
by the trade, it seems a bit strange that I would be using Python to write my Javascript
when I’m a perfectly comfortable Javascript developer why am I using this obscure Python
library in order to make interactive web page as soon as Where will the main reason is I work for these
guys, Decision Lab, we are a mathematical modelling consultancy based in Central London. On that point about being in Central London,
if you are following the latest with the whole Brexit debacle I’m terribly sorry! It’s quite embarrassing these days. I have no idea what’s going on either. In fact I think it’s just worth saying this
is my third European Pycon in as many weeks, I really do love the Python and opensource
community, I think it’s brilliant when we have so many people from different nationalities
and backgrounds all coming together for technology sake. More of this, less of that Brexit nonsense
please. So moving on! As I said Decision Lab, we are a mathematical
modelling consultancy, our expertise are in machine learning, optimisation and simulation. We work in defence, engineering, intelligence
and a few other public body organisations. Our main selling point is that we enjoy interesting
and unusual problems and datasets. We also, rather than write reports for our
clients we tend to build tools and project to solve problems, we don’t settle with writing
a single piece of data analysis, we want to make a tool that’s useful for ongoing analysis. Sop recent examples of projects are you can
see here we developed a system to detect illegal gold mining in the Colombian Amazonian rain
forest using machine learning with human in the loop and at the moment we’re also developing
a predictive maintenance service for the type 45 Royal Navy destroyer, so you can forget
AWS as our deployment is currently a warship, so it presents interesting problems. So we’re a company of data scientists and
a smaller team of software engineers as well, one of the things we notice in our team or
we come up against sometimes is that there is in a sense two cultures to Python. It has the data science culture and background
or pydata and also the software engineering side of Python, the database and web side. The classic two cultures you’d have from CP
snow in the late 50s and so on, but broadly speaking there are two kinds of background
of people that come to Python is significant, these people have different priorities and
different sorts of approaches and collaborations between the two groups is key. But that can also be costly sometimes on small
projects if you want someone who maybe just finished their PhD and data science related
field to suddenly become a software engineer, that takes time and is expensive, so my team
have bridged that gap, the production team we aim to take our tools from data scientists
and turn them into products, but when we’re developing alphas or proof of concepts, we
really want to be able to empower a data scientist and minimise the new technology that someone
who might just finish their PhD or whose background is in simulation modelling, minimise those
new technology that is they need to learn or start to use, in order to be able to build
something effective and useful for us. So I want my data scientist to take the prototyping
proof of concept dashboard. If they are developing a website a proof of
concept dashboard they are not going to be in web development, so we found this tool
by Plotly called dash. It’s an opensource product, there are some
paid consultancy options, I think you can sponsor a developer and so on. Recently, it’s a young project, recently it
had version 1 in June, it does change rapidly. So the unique selling point of the tool is
that you can build an interactive website without having to use Javascript, that’s mostly
a good thing. Sometimes, not always. We’ll explore that in the rest of the talk. So we’ll go over a little example of how Dash
works, explain some of the things you can do with Dash quickly, some of the tips and
perhaps good practice that is we’ve start to learn and some of the things we’re trying
to take forward and then give an opinion about when you probably need top stop using Dash
and actually start building a full scale web application. There are some code examples in the talk and
you’ll see them at the Github repository there, I’ll put it up and tweet it out. First things forward, just pip install Dash
and you can start running your applications, I ran out of time before this talk but I am
convinced you can actually write a one line Dash App based on the brilliant lightning
talk we had last night so I’ll figure that out. So it was a brief hello world example. Here it is, just to talk us through. We have two libraries here we have to import,
Dash and dash.html.components I will come to what they do in a minute. We create the application, if you are familiar
with Flask it’s exactly the same, in fact under the hood Dash is creating a Flask application
for you and you can supply or inject your own Flask application if you want to do something
a bit more advanced. Here we define a layout and we have a simple
layout here with just a single division and inside there we have a header tab or heading
1, and then we see down here we literally are running our application. So he bravely loads up the live code example
… so here you can see, this is the Github repository we’ve got and now we’ve got a very
basic hello world example running on my laptop locally and there’s no need to use any Javascript,
you create a single web application. I have tweaked it a little bit and put my
logo up at the top, that’s because you are aye able to inject style without having to
declare it in the application, it will loo look for that automatically, and I’ve also
in that Github repository there’s a Docker imposed file, if you run that on various ports
you can have a play around with them. So we’ll come back to that in a second. That’s great, but let’s discuss what’s happening
under the hood internally. As I said before there are two packages that
are being used internally by Dash here. The first one is Dash HTML components, that’s
really just a wrapper, Dash created Dash components which are Python callables that you can use
to create a web layout. Under the hood is just wraps all these different
React components which themselves wrap or create HTML components based on the HTML schema. There’s also the other package Dash which
manages the relationships between the different components and it references the application
using the Flask instance for you. But the title of the talk is to do with interactive
web pages so we want to have an interactive Dash application rather than hello world. So the first application I’ll call hello anyone
application and broadly speaking what it will do is this, we’re going to say hello to someone
but take their name using a text box. So before we discuss how to do that in Dash
let’s look at how that works in plain old Javascript, so first off we’ll need some HTML,
to make it work we need three DOM nodes, it’s not because I’m Dom and I’m really vain, it’s
because it’s Document Object Model, which is a fundamental part of web development,
I just happen to be a web developer called Dom which is useful sometimes! So these three DOM nodes let’s look, you’ve
got the paragraph which is going to contain the text, you have a span inside the paragraph
that will contain the name of the person we’re going to greet and then we also have an input
here which is like a text box, so we’re giving the span and input Ids which are denoted by
the #there. Our Javascript application is going to take
the value from the input, hello input and set the HTML inside the span to be a value
from that particular input. But crucially rather than doing that once
and forget about it it is going to have to monitor it continually because it’s a Javascript,
so each key press they make we are going to have to update every single time that happens. So ordinarily with a Javascript application
the technologies at work here are Javascript and React, Javascript runs in the browser
and allows you to alter the DOM, manipulate it to do whatever you want. There’s also React that means that rather
than having to write complex Javascript applications to continually monitor things or constantly
access different parts of the DOM we can just declare using a declarative interface, declare
what changes we want to happen and React will sort the rest out for us. So in a standard plain old Javascript application,
React application we’ll have essentially a function that looks like this, which will
interpolate our string for us, so it will take the value from that particular text box,
we need to put it inside our spam. So in React that code that defines what we
want to happen lives inside the browser, it’s a Javascript function that’s going to be executed
inside the browser on the user’s computer. However in Dash all of these interactions
between different parts of the DOM are handled by what we call callbacks, these are Python
functions which live on the server instead so whenever the inputs change they’re executed
on the server that means we have to cross a network, go from browser to server and back
again every time we have some kind of interactive change or thing happen inside of our application. So let’s look at what this particular App
would look like inside of Dash, so you can see here again we are defining our layout
at the top here, this time we have our heading, just like we had before, we’ve got our input
as well which will be the text box that takes the user’s name, and here we define this particular
callback so what we are saying here, you can see it’s decorator at work, so we are saying
in this particular callback, whenever the value inside our input changes, that’s the
input is callback, we want to run this piece of code which is going to interpolate our
string for us and then we want to send the output of this particular callback and put
it or set it as the value inside of that particular heading that’s working at the top here. So this means essentially that Dash is going
to create a React application for us and whenever that React application notices that something
inside of that text box changed it’s going to go to the server say hey what do I do now
and the server will say oh update the heading to give the user this particular value from
our function. So it’s a bit like Excel really, whenever
you say this particular cell will be the input or it’s going to be this cell multiplied by
this cell, Excel will work out the relationships between the different cells and call all the
functions as required. So this means now every single change that
happens inside our web application needs to run a Python function which lives on the server. This really does cause a large performance
loss, Javascript and React in particular is lightning fast and has a very cool string,
string algorithm called fibre, which is one of the reasons it’s such a popular tool, now
we’re constantly crossing a network rather than exploiting the benefits of Javascript
inside the browser. So it’s a big performance loss there, but
that’s actually okay. It’s not something I’ve ever had to really
worry about, I want to empower a data scientist to build an App really quickly using familiar
technologies in order to get to proof of concept phase, I’m not worried about production workload
or whether or not it will crash if we have hundreds of users, or if it’s possibly a little
bit laggy, as a especially application it’s for data scientists and has a specialist audience. Admitting we are going to maybe take a hit
on performance but allow the data scientist to go really far without having to hand over
work to software engineers, I want to look at some of the things you can do in Dash really
easily or quickly. So as the name suggests it’s mostly geared
towards dashboards, but there are other tools as well that do that kind of thing, Panel
and Bouquet and so on forth. But the selling point of Dash is you can do
transactional things really easily as well. Let’s have a look at some other quick examples
and these are all in the Github feed you be see the URL for there. I’ll list a dataset here, the Titanic passengers
dataset, a couple of hundreds rows, display a graph and make a transactional request to
show why the Dash is powerful. First example first the Titanic passenger
dataset, again here we’re defining a layout that we’re going to use and the first main
component we’re using is a dash table, this is basically going to display our data frame
for us with pagination if we need it and is going to put columns in and so on and so forth. And we’ve also got this drop down, so I decided
rather than display the dataset I’m going to allow a use tore filter based on a particular
category, so is that will be done by a callback we’re defining here, so we’re monitoring that
particular drop down and alter the data that gets sent to that Dash table as a result. Here we’re just literally filters based on
sex you can see a standard filter there. — sorry I didn’t realise I didn’t show some
of the examples before. So just to show some of the examples we have
here and prove that this works. Here we’ve got the basic example we had before,
so hello Pycon and you can see we have an interactive web application there, absolutely
no Javascript involved. Sorry I should have shown that to you before. Now if we come to our more involved example
with the Titanic dataset, you can see here I’ve just listed, this is running live on
my laptop at the moment, I’ve loaded a dataset in, from Pd.csv fed to the Dash using a callback,
now it’s showing all passengers, we can pick female passengers then it will constantly
monitor that for me as well. You can see this is a really super basic example,
but with a couple of those callbacks combined together, some other interfaces and a bit
of thought you can make advanced applications without ever touching any client side code. Next thing I want to do is display a graph,
so this is a project by Plotly who are obviously famous for their graphing tools so it’s no
surprise that actually the graphing tools that we get build in essentially to dash are
really quite strong and powerful. Here we’re defining a layout, this is the
second example, we’ve got you can see here it looks like a lot of code but that’s because
it’s formatted with black, but you can see there’s this component here from Dash, core
component which is just the simple straightforward graph, it has an interface which is similar
to a standard Plotly graph that you might use in notebook or something like that, and
here at the top you can see I have just provide it had with a basic dataset based on Titanic
passengers. So this is letters of the alphabet and frequency
with which passengers on the Titanic had a first name beginning with that letter. So I’ve split up or counted the passengers
on the Titanic by the first letter of the first name. Also because this is for a bit of fun I decided
to do it in one line because I always find that more fun than coding things to be readable. So you can have a look in the repository there,
I’m sure there’s a more efficient way to do that even in one line. So to demonstrate what that looks like now. So you can see here on the next example and
we’ve produced add very basic histogram, I think the most common letter was J for passenger
to have, so here we’ve created a web application that is displaying data in a chart, on a user’s
browser with no Javascript involved whatsoever and you can use a callback to filter sex like
I did on the dash table example, or you can use any other combination of different dash
tools together to make something more involved and impressive. So the next thing I want to demonstrate is
a transactional web application, so this is a classic web development example here making
a to-do application. And so just very basic we take some input
from a user, store it somewhere and display as a list of tasks. So to do that you can see we have an ordered
list, I have create this had dash layout here, a heading at the top and ordered list that
will list all of the different tasks I have in the data store somewhere. We have an input and also take the text for
a new task and also got a button. So that button is just going to be our submit
for a form. So here I’m storing, I’ve created a basic
data store, in this case is just a dictionary in memory, but there’s no reason it couldn’t
be post res or call an API or do some other magic. Here it’s a basic function that that’s going
to return each task from my data store wrapped in the Dash component, which is a list item. So here again this might look like a lot of
code but it’s all stuff that we’ve gone through before, we’ve got this decorator, here we
are going to say whenever this particular button is called or whenever the button is
clicked, i.e. the number of times it’s clicked increases by one, we are going to send something
to our list, where the tasks are being displayed. Here this state option means whenever the
button is clicked I want to get the value from my input, from my text box, but I don’t
want to call this particular function every time that text box changes, so I’m getting
the state of the text box rather than monitoring it for the changes. So here you’re appending it to a particular
list inside of that dictionary and then you can see the results. So the example we have here, we can do something
like this … you can see this persists when I refresh the page because it’s stored in
my memory on my computer but you can put that inside the database or do something else,
that ability to change data and persist it I think is really powerful and one of the
things that makes dash so good really and you can see some of the reasons why in an
example I’ll show you in a minute. So now you can use that particular functionality
in Dash to go really really far, so an example of something recently was the gold mining
project that I mentioned earlier. We were able to build a very basic, this is
actually a couple of versions behind where it is at the moment, but a very basic interactive
application released by Colombian law enforcement, it was built by one junior and one middle,
mid career data scientist over the course of a couple of months and that was as they
were doing machine learning, doing model training and so on. It wasn’t built by software engineers or handed
over to a big UI team, it was literally built by the data scientists liaising and working
with people on the project. Here you can see we are able to use human
in the loop machine learning using Dash, because we can take inputs from a user and store values
inside the database which inform our model, which is used to take our satellite imagery
and so we are not just displaying data, we are able to manipulate it and alter it using
dashboard tool. So we’ve taken, dash quite far and started
to build much larger, dash applications and really start to take it as far as we possibly
can, because it’s such a useful tool to accelerate some of our development. So some of the standards tips for any larger
project the main thing is organise your code and that partly comes down to the two cultures
of Python that I mentioned before, because some of these tips are splitting up callbacks
from layouts or giving individual features their own functions and directories and so
on and abstracting that logic isn’t necessarily something that comes immediately to somebody
who might have a background working in data science. So standard layout that we might have here,
we just run the script as a module using DOM domain then we have callbacks and layouts
split up, separate from the App as well. One thing the project we were also working
on at the moment is to take some of the code that we have to manage building applications
in Dash and build a pseudo framework using it so at the moment we’ve actually created
a mini framework internally that uses a decorator syntax in order to define layouts and callbacks
and brings them together at one time and builds an application and implements things like
routine and navigation. If this is something you might be interested
in, as we’re looking to opensource this project, please do get in touch, in fact I have had
a bit of interest from other people as well, it would be nice to get a mini team going
on that. It brings me now to the final few points about
when to stop using Dash and to start building what we might call a proper web application. So you have to think about what Dash is good
for? So as I said one of the brilliant things about
Dash is that it allows for really rapid UI development by people who aren’t specialists
in that field, a data scientist can develop a full UI straight away without having to
become an expert in Javascript or associated technologies on the web like CSS and so on. And it really allows for informed web development
as well, that your user interface can be built by people who know their stuff on machine
learning and know the problem they are building a tool for really inside out and don’t have
to handover work necessarily to a software engineer or Javascript developer who doesn’t
necessarily live in the world or have that interest. But also presents problems for exactly the
same reasons. So for example that rapid UI development that
you benefit from so much towards the start of after project, if you know you are going
to be developing a much bigger tool are you going to be creating technical debt straight
away? Do you need to employ a Javascript developer
or get one on your team from the get go? And this idea that UI development is also
informed, but informed by whom? Data scientists, that’s great but they are
not users, do you need to get a user researcher in or employ a UX consultant to the think
how do to build a good App rather than just an App. Dash is good for that, investing a bit of
time and organising the code allows you go really far inside the Dash application but
reassuring front end developers will still have a job, so it’s not going to replace Javascript,
so thank you very much for listening. Dam I thought I got rid of all the references
to Berlin, but there’s one at the end! Sorry. Time for a question.>>This is possibly a stupid question, I’m
just wondering for visualisation that makes sense, can you also use it to write back to
a data frame or database? MR WELDON: The question was you can use Dash
for visualisation but can you also use it to write back and persist changes, yes, you
can. So for example on the gold mining App that
we had there, we’ll be using human in the loop machine learning and actually taking
the project a Dash application and storing it on our server, and the to do application
for example here … So these tasks now are stored inside of my
computer’s memory, there’s no reason why I couldn’t necessarily persist that to a database
inside of my callback or make an API call or something. So when I refresh the page here they come
back and I could have had something else in and that’s now stored on my server. So yeah, you can. Thank you. [APPLAUSE]

Leave a Reply

Your email address will not be published. Required fields are marked *