Monday, September 11, 2017

More heatmaps: other types of crime is San Francisco, 2003-2016

This is a follow-up to the earlier post about visualizing SFPD data. Now let's focus on other crime categories: vehicle theft, larceny/theft, burglary, assault.

Animated GIF is not the best way to explore patterns. Check out the original interactive charts for each type of crime below.

SFPD incident database and heatmaps: drug-related incident patterns

San Francisco Police Department provides free access to a great data source: The following couple of charts display drug-related incident distribution by year, weekday, time of day, and district.

Some observations:

  • the notorious "welfare Wednesday" effect is present in early years;
  • incident numbers drop significantly starting 2010, which may be caused by Proposition 215: Wikipedia says "It was not until March 2009 that federal officials announced that they would no longer try to thwart medical marijuana distribution/use in California".
There is a follow-up post focusing on other types of crime in SF using the same data source.

The following are the steps to create the weekday/time heatmap chart using editor.

Tuesday, August 15, 2017

World Health Organization (WHO): spotting spending patterns in 2000-2015

It took me a while to find and groom WHO spending numbers. All financial reports I used can be found here. There were a few challenges, here are some worth mentioning.

1. Between 2000 and 2011, the WHO published two-year financial reports, and in 2012, they switched to one-year schedule.

2. The list of expenditure categories was changing from year two year. As a result, I could not break down equipment and supplies category into meaningful groups, so medical supplies and furniture fall into the same category. Same with the "Other" category: it includes grants, research contracts, local subsidies and what not.

3. Since 2008, WHO financial reports stopped providing staff costs breakdown, so "Stuff costs" category includes full-time and part-time employees, consultants and even governing body delegates.

Below is the two-axis chart that shows absolute spending numbers as bars and the share of some spending categories as lines:

Key observations:

  • the WHO is drifting towards outsourcing some of the activities: watch the staff+equipment spending go down while contractual services going up;
  • the share of travel expenses grew three times between 2000 and 2015.

It turned out both observations make sense: just google "world health organization outsourcing" and "world health organization travel expenses".

Tuesday, August 8, 2017

Animated GIF is alive and well

I have just came across a blog featuring high-quality charts visualizing all kinds of data: Charts are made with, highcharts, bokeh, and other tools. Some charts are published as high-quality animated GIFs. Apparently, the blogger uses this data format as a last resort - not so many tools out there support animation right out of the box. Anyways, good to see in a good company.

Log scale for XY charts

From now on, supports log scale in all XY charts (bubble, scatter, motion). Consider the following visualization of World Bank and SIPRI Military Expenditure Database data:

Why log scale? There are a few cases when a log scale is appropriate. The most well known is: log scales allow a large range to be displayed without small values being compressed down into bottom of the graph. Consider the same visualization made using linear scale (this is just a screenshot, not an interactive chart, but it gives you a good idea about the problem):

Tuesday, August 1, 2017

More fun with motion charts: European top 20 imports and exports, 1990-2016

World Bank Group provides a lot of data to play with. Let's build a motion chart based on export, import, and GDP data for European top 20 economies, grouped by regions - Northern, Western, Eastern, and Southern.

Watch international trade shrinking between 2008 and 2009!

Monday, July 31, 2017

Motion charts: add categories and animate those bubbles now supports motion charts. Motion charts are basically bubble charts with one more dimension added (usually, it's time). In, we use category for this dimension, remember category radio buttons in pie charts and bar charts? So, our motion charts are bubble charts with category radio buttons.

Recently, Statistics Canada published a quick overview of interprovincial migration data, using simple line charts. Let's make this data look fancier:

Here are the steps to create this motion chart.

Thursday, June 29, 2017

Happy Birthday, Canada: animated categories

While Canadians looking for the ultimate party to celebrate Canada's 150th birthday on July 1, we will have a quick look at their spending habits. And we will do that by using new feature - animated categories.

Chart options panels went through some reorganization. Now we have "Current category label" panel:

and "Animation" panel:

For charts with multiple data rows that are considered categories (pie, regular bar and comparative histogram charts), can walk through all categories and update the appearance, creating an animation effect - see chart below.

Try it for yourself - create a pie, regular bar or comparative histogram chart with multiple data rows and play with animation settings. The feature comes handy when you embed visualization into a presentation and do not have a convenient way to click or hover over the chart to navigate between categories.

Wednesday, June 14, 2017

The case for pie charts

Pie charts get a lot of criticism these days. I must agree that bar charts deliver the message better because:

  • bar charts can present data for multiple categories without user interaction;
  • human eye is much better at telling differences between lengths than it is at determining differences between areas.

The charts below visualize the same data, and they all have the same size of 640x640 pixels. Decide for yourself which one does it better.

Pie chart in question - move your mouse or tap over radio buttons to change category:

Thursday, June 1, 2017

Scalable charts

Great news: charts are now fully scalable. Content editors and users can change the size of the container - the chart will scale accordingly, preserving the original aspect ratio, without losing any functionality or quality.

Source code for this test HTML file - pay attention to the 100% width style:

      <embed src="example-drinking.svg"  type="image/svg+xml" style="width:100%;" />

To see this in action, open and play with the browser window size - you will see sample charts shrinking and expanding as well.

Tuesday, May 23, 2017

Integration API: charts with live data integration API comes handy when you want your published chart to display up-to-date information and you do not have the luxury of logging in to the editor, tweaking the data and re-publishing the chart manually every time the data changes. Consider the following P/E ratio vs. Price to book bubble chart that we want to be updated every hour:

Tuesday, May 9, 2017 number formatting

Users want to have the ability to tweak number formatting. No, users need this feature. Just imagine having labels like "1382323332" and "1340152688" on a diagram comparing Chinese and Indian population. Or imagine a US GDP chart with a "$20000000000000" mark on the vertical axis. Or imagine that your audience wants to see negative amounts of some specific currency: "-£349.3300". supports number formatting for axis marks, data labels and callouts:

Since the rules of number formatting can be complicated, we decided not to invent our own mechanism for it. We just use the standard instead: ECMAScript Internationalization API Specification. It's a long specification, but the part that we are interested in boils down to this Mozilla document. This document defines how Javascript developers can specify the locale and formatting options when converting numbers to strings. For example, an object that enables Japanese yen formatting can be instantiated as follows:

new Intl.NumberFormat('ja-JP', { style: 'currency', currency: 'JPY' });

and it can produce strings like "¥123,457". does not provide any programming platform for chart creators, but it allows to pass the locale 'ja-JP' and the option object { style: 'currency', currency: 'JPY' } as additional parameters to the {value} keyword used in axis marks, data labels and callouts. The convention is to extend the "value" keyword with a semicolon followed by the number format specification:

NumberFormat:{locale:'<desired locale>', options:<desired options>}

For example:

{value;NumberFormat:{locale:'en', options:{useGrouping: true }}} will produce numbers with comma as a thousands separator

{value;NumberFormat:{locale:'en-US',options: style:'currency',currency:'GBP',minimumFractionDigits:4}}} will produce British pound amounts with 4 digits after decimal point.

In the example below, the callout format

{series}: {value;NumberFormat:{locale:'en', options:{useGrouping: true }}}

is specified, which makes the callout for the "0-4" age group with value 1453708 to be formatted as "0-4: 1,453,708":

Sunday, May 7, 2017

Comparative histogram charts: population by age group

Great news: now supports comparative histograms. This kind of charts comes  handy when you want to compare two or more groups of series side-by-side. In most cases, this type of charts is used to draw a population pyramid, but it has other uses as well.

The following chart shows country population by age group, for multiple years, comparing male and female population.

This post discusses comparative histogram creation process step-by-step.

Friday, January 6, 2017

US and Canada demographics: immigration, temporary workers, refugees in 1935-2014

I was curious about the differences between American and Canadian immigration systems. After 2016 US elections, we hear a lot about those differences, but I could not find a good source with actual numbers compared. Let's do it here. As usual, here is the Google Docs spreadsheet with all relevant data. As usual, all charts below are interactive, just move your mouse over the line or the legend item.

Let's start with the simplest part - total population numbers provided by Wikipedia:

No surprises here - these countries are economically and culturally close, so population growth patterns are similar.

Let's have a look at natural population growth numbers pulled from UNDESA (UN Department of Economic and Social Affairs) and The World Bank: