Skip to content

Instantly share code, notes, and snippets.

@mms-marko
Forked from kerryrodden/.block
Last active September 21, 2015 20:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mms-marko/25128bf379c2eaffc97c to your computer and use it in GitHub Desktop.
Save mms-marko/25128bf379c2eaffc97c to your computer and use it in GitHub Desktop.
Sequences sunburst

This example shows how it is possible to use a D3 sunburst visualization (partition layout) with data that describes sequences of events.

A good use case is to summarize navigation paths through a web site, as in the sample synthetic data file (visit_sequences.csv). The visualization makes it easy to understand visits that start directly on a product page (e.g. after landing there from a search engine), compared to visits where users arrive on the site's home page and navigate from there. Where a funnel lets you understand a single pre-selected path, this allows you to see all possible paths.

Features:

  • works with data that is in a CSV format (you don't need to pre-generate a hierarchical JSON file, unless your data file is very large)
  • interactive breadcrumb trail helps to emphasize the sequence, so that it is easy for a first-time user to understand what they are seeing
  • percentages are shown explicitly, to help overcome the distortion of the data that occurs when using a radial presentation

If you want to simply reuse this with your own data, here are some tips for generating the CSV file:

  • no header is required (but it's OK if one is present)
  • use a hyphen to separate the steps in the sequence
  • the step names should be one word only, and ideally should be kept short. Non-alphanumeric characters will probably cause problems (I haven't tested this).
  • every sequence should have an "end" marker as the last element, unless it has been truncated because it is longer than the maximum sequence length (6, in the example). The purpose of the "end" marker is to distinguish a true end point (e.g. the user left the site) from an end point that has been forced by truncation.
  • each line should be a complete path from root to leaf - don't include counts for intermediate steps. For example, include "home-search-end" and "home-search-product-end" but not "home-search" - the latter is computed by the partition layout, by adding up the counts of all the sequences with that prefix.
  • to keep the number of permutations low, use a small number of unique step names, and a small maximum sequence length. Larger numbers of either of these will lead to a very large CSV that will be slow to process (and therefore require pre-processing into hierarchical JSON).

I created this example in my work at Google, but it is not part of any Google product. It is covered by the Apache license:

Copyright 2013 Google Inc. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Sequences sunburst</title>
<script src="http://d3js.org/d3.v3.min.js"></script>
<link rel="stylesheet" type="text/css"
href="https://fonts.googleapis.com/css?family=Open+Sans:400,600">
<link rel="stylesheet" type="text/css" href="sequences.css"/>
</head>
<body>
<div id="main">
<div id="sequence"></div>
<div id="chart">
<div id="explanation" >
<span id="percentage"></span><br/>
<span id="orderCount"></span><br/>
<span id="orderMsg">Path taken by Orders through ICS.</span>
</div>
</div>
</div>
<div id="sidebar">
<input type="checkbox" id="togglelegend"> Legend<br/>
<div id="legend" style="visibility: hidden;"></div>
</div>
<script type="text/javascript" src="sequences.js"></script>
<script type="text/javascript">
// Hack to make this example display correctly in an iframe on bl.ocks.org
d3.select(self.frameElement).style("height", "1000px");
</script>
</body>
</html>
body {
font-family: 'Open Sans', sans-serif;
font-size: 12px;
font-weight: 400;
background-color: #fff;
width: 1000px;
height: 950px;
margin-top: 10px;
}
#main {
float: left;
width: 750px;
}
#sidebar {
float: right;
width: 100px;
}
#sequence {
width: 600px;
height: 70px;
}
#legend {
padding: 10px 0 0 3px;
}
#sequence text, #legend text {
font-weight: 600;
fill: #fff;
}
#chart {
position: relative;
}
#chart path {
stroke: #fff;
}
#explanation {
position: absolute;
top: 370px;
left: 355px;
width: 200px;
text-align: center;
color: #666;
z-index: -1;
}
#percentage {
font-size: 3.0em;
}
// Dimensions of sunburst.
var width = 900;
var height = 850;
var radius = Math.min(width, height) / 2;
// Breadcrumb dimensions: width, height, spacing, width of tip/tail.
var b = {
w: 90, h: 30, s: 3, t: 10,
// width as a function of step.
stepWidth: function(d) { return d.name.charAt(0) != "!" ? b.w : (b.w*3); }
};
// Mapping of step names to colors.
/*
var colors = {
"standard": "#0026FF",
"customized": "#EACF81",
"sms.com": "#FF7FED",
"providerTech": "#0094FF",
"validXML": "#000000",
"invalidXML": "#000000",
"invalidXML": "#000000",
"Missing Data": "#000000",
"Invalid Data": "#000000",
"icsAccept": "#f5cd05",
"icsReject": "#980000",
"OK": "#1acc34",
"ERROR": "#980000",
"!": "#999999"
};
*/
// http://colorbrewer2.org/
var colors = {
"standard": "#023858",
"customized": "#ae017e",
"sms.com": "#0570b0",
"providerTech": "#74a9cf",
"validXML": "#a1d99b",
"invalidXML": "#feb24c",
"Missing Data": "#252525",
"Invalid Data": "#bdbdbd",
"icsAccept": "#41ab5d",
"icsReject": "#fc4e2a",
"OK": "#006d2c",
"ERROR": "#bd0026",
"!": "#800026"
};
function stepColor(d) {
// all detailed errors will use the same color.
var c = colors["!"];
if ( d.name.charAt(0) != "!" ) {
c = colors[d.name];
}
return c;
};
// Total size of all segments; we set this later, after loading the data.
var totalSize = 0;
var vis = d3.select("#chart").append("svg:svg")
.attr("width", width)
.attr("height", height)
.append("svg:g")
.attr("id", "container")
.attr("transform", "translate(" + width / 2 + "," + height / 2 + ")");
var partition = d3.layout.partition()
.size([2 * Math.PI, radius * radius])
.value(function(d) { return d.size; });
var arc = d3.svg.arc()
.startAngle(function(d) { return d.x; })
.endAngle(function(d) { return d.x + d.dx; })
.innerRadius(function(d) { return Math.sqrt(d.y); })
.outerRadius(function(d) { return Math.sqrt(d.y + d.dy); });
// Use d3.text and d3.csv.parseRows so that we do not need to have a header
// row, and can receive the csv as an array of arrays.
d3.text("visit-sequences.csv", function(text) {
var csv = d3.csv.parseRows(text);
var json = buildHierarchy(csv);
createVisualization(json);
setDefaultExplanation();
});
// Main function to draw and set up the visualization, once we have the data.
function createVisualization(json) {
// Basic setup of page elements.
initializeBreadcrumbTrail();
drawLegend();
d3.select("#togglelegend").on("click", toggleLegend);
// Bounding circle underneath the sunburst, to make it easier to detect
// when the mouse leaves the parent g.
vis.append("svg:circle")
.attr("r", radius)
.style("opacity", 0);
// For efficiency, filter nodes to keep only those large enough to see.
var nodes = partition.nodes(json)
.filter(function(d) {
return (d.dx > 0.005); // 0.005 radians = 0.29 degrees
});
var path = vis.data([json]).selectAll("path")
.data(nodes)
.enter().append("svg:path")
.attr("display", function(d) { return d.depth ? null : "none"; })
.attr("d", arc)
.attr("fill-rule", "evenodd")
.style("fill", stepColor )
.style("opacity", 1)
.on("mouseover", mouseover);
// Add the mouseleave handler to the bounding circle.
d3.select("#container").on("mouseleave", mouseleave);
// Get total size of the tree = value of root node from partition.
totalSize = path.node().__data__.value;
};
// Fade all but the current sequence, and show it in the breadcrumb trail.
function mouseover(d) {
// console.log(d);
// errorCounts will be treated specially since Orders can have many errors.
var orderCount = d.isError ? d.errorCount : d.value;
var percentage = (100 * orderCount / totalSize).toPrecision(3);
var percentageString = percentage + "%";
if (percentage < 0.1) {
percentageString = "< 0.1%";
}
var orderCountString = "(" + Math.round(orderCount) + " of " + Math.round(totalSize) + " Orders)";
d3.select("#percentage")
.text(percentageString);
d3.select("#orderCount")
.text(orderCountString);
d3.select("#explanation")
.style("visibility", "");
var sequenceArray = getAncestors(d);
updateBreadcrumbs(sequenceArray, percentageString);
// Fade all the segments.
d3.selectAll("path")
.style("opacity", 0.3);
// Then highlight only those that are an ancestor of the current segment.
vis.selectAll("path")
.filter(function(node) {
return (sequenceArray.indexOf(node) >= 0);
})
.style("opacity", 1);
}
// Restore everything to full opacity when moving off the visualization.
function mouseleave(d) {
// Hide the breadcrumb trail
d3.select("#trail")
.style("visibility", "hidden");
// Deactivate all segments during transition.
d3.selectAll("path").on("mouseover", null);
// Transition each segment to full opacity and then reactivate it.
d3.selectAll("path")
.transition()
.duration(1000)
.style("opacity", 1)
.each("end", function() {
d3.select(this).on("mouseover", mouseover);
});
setDefaultExplanation();
}
function setDefaultExplanation() {
d3.select("#percentage")
.text("100%");
d3.select("#orderCount")
.text(Math.round(totalSize) + " Orders were sampled.");
}
// Given a node in a partition layout, return an array of all of its ancestor
// nodes, highest first, but excluding the root.
function getAncestors(node) {
var path = [];
var current = node;
while (current.parent) {
path.unshift(current);
current = current.parent;
}
return path;
}
function initializeBreadcrumbTrail() {
// Add the svg area.
var trail = d3.select("#sequence").append("svg:svg")
.attr("width", width)
.attr("height", 50)
.attr("id", "trail");
// Add the label at the end, for the percentage.
trail.append("svg:text")
.attr("id", "endlabel")
.style("fill", "#000");
}
// Generate a string that describes the points of a breadcrumb polygon.
function breadcrumbPoints(d, i) {
var width = b.stepWidth(d);
var points = [];
points.push("0,0");
points.push(width + ",0");
points.push(width + b.t + "," + (b.h / 2));
points.push(width + "," + b.h);
points.push("0," + b.h);
if (i > 0) { // Leftmost breadcrumb; don't include 6th vertex.
points.push(b.t + "," + (b.h / 2));
}
return points.join(" ");
}
// Update the breadcrumb trail to show the current sequence and percentage.
function updateBreadcrumbs(nodeArray, percentageString) {
// Data join; key function combines name and depth (= position in sequence).
var g = d3.select("#trail")
.selectAll("g")
.data(nodeArray, function(d) { return d.name + d.depth; });
// Add breadcrumb and label for entering nodes.
var entering = g.enter().append("svg:g");
entering.append("svg:polygon")
.attr("points", breadcrumbPoints)
.style("fill", stepColor);
entering.append("svg:text")
.attr("x", function(d) { return (b.stepWidth(d) + b.t) / 2; } )
.attr("y", b.h / 2)
.attr("dy", "0.35em")
.attr("text-anchor", "middle")
.text(function(d) { return d.name; });
// Set position for entering and updating nodes.
g.attr("transform", function(d, i) {
return "translate(" + i * (b.w + b.s) + ", 0)";
});
// Remove exiting nodes.
g.exit().remove();
var percentSignX = 0.5 * (b.w + b.s);
for (var i = 0; i < nodeArray.length; i++) {
percentSignX += (b.stepWidth(nodeArray[i]) + b.s);
}
// Now move and update the percentage at the end.
d3.select("#trail").select("#endlabel")
// .attr("x", (nodeArray.length + 0.5) * (b.w + b.s))
.attr("x", percentSignX)
.attr("y", b.h / 2)
.attr("dy", "0.35em")
.attr("text-anchor", "middle")
.text(percentageString);
// Make the breadcrumb trail visible, if it's hidden.
d3.select("#trail")
.style("visibility", "");
}
function drawLegend() {
// Dimensions of legend item: width, height, spacing, radius of rounded rect.
var li = {
w: 75, h: 30, s: 3, r: 3
};
var legend = d3.select("#legend").append("svg:svg")
.attr("width", li.w)
.attr("height", d3.keys(colors).length * (li.h + li.s));
var g = legend.selectAll("g")
.data(d3.entries(colors))
.enter().append("svg:g")
.attr("transform", function(d, i) {
return "translate(0," + i * (li.h + li.s) + ")";
});
g.append("svg:rect")
.attr("rx", li.r)
.attr("ry", li.r)
.attr("width", li.w)
.attr("height", li.h)
.style("fill", function(d) { return d.value; });
g.append("svg:text")
.attr("x", li.w / 2)
.attr("y", li.h / 2)
.attr("dy", "0.35em")
.attr("text-anchor", "middle")
.text(function(d) { return d.key; });
}
function toggleLegend() {
var legend = d3.select("#legend");
if (legend.style("visibility") == "hidden") {
legend.style("visibility", "");
} else {
legend.style("visibility", "hidden");
}
}
// Take a 2-column CSV and transform it into a hierarchical structure suitable
// for a partition layout. The first column is a sequence of step names, from
// root to leaf, separated by hyphens. The second column is a count of how
// often that sequence occurred.
function buildHierarchy(csv) {
var root = {"name": "root", "children": []};
for (var i = 0; i < csv.length; i++) {
var sequence = csv[i][0];
var size = +csv[i][1];
if (isNaN(size)) { // e.g. if this is a header row
continue;
}
// concrete errors will also include a count of the error type.
var errorCount = +csv[i][2];
var parts = sequence.split("-");
var currentNode = root;
for (var j = 0; j < parts.length; j++) {
var children = currentNode["children"];
var nodeName = parts[j];
var childNode;
if (j + 1 < parts.length) {
// Not yet at the end of the sequence; move down the tree.
var foundChild = false;
for (var k = 0; k < children.length; k++) {
if (children[k]["name"] == nodeName) {
childNode = children[k];
foundChild = true;
break;
}
}
// If we don't already have a child node for this branch, create it.
if (!foundChild) {
childNode = {"name": nodeName, "children": []};
children.push(childNode);
}
currentNode = childNode;
} else {
// Reached the end of the sequence; create a leaf node.
childNode = {"name": nodeName, "size": size, "isError":false};
if (!isNaN(errorCount)) {
childNode.isError = true;
childNode.errorCount = errorCount;
}
children.push(childNode);
}
}
}
return root;
};
We can make this file beautiful and searchable if this error is corrected: It looks like row 7 should actually have 2 columns, instead of 3. in line 6.
standard-sms.com-invalidXML-Missing Data-!Order Line Block,4
standard-sms.com-invalidXML-Invalid Data-!Email address,4
standard-sms.com-invalidXML-Invalid Data-!Insurance name,2
standard-sms.com-invalidXML-Invalid Data-!E1 Part number,1
standard-sms.com-invalidXML-Invalid Data-!Order line quantity,1
standard-sms.com-validXML-icsAccept-OK,952
standard-sms.com-validXML-icsAccept-ERROR-!Duplicate Line,34,60
standard-sms.com-validXML-icsAccept-ERROR-!EZ SHIP PATIENT,10,17
standard-sms.com-validXML-icsAccept-ERROR-!Insurance Invalid Hierarchy primary,14,23
standard-sms.com-validXML-icsAccept-ERROR-!Insurance Missing,64,105
standard-sms.com-validXML-icsAccept-ERROR-!Invalid INSURANCE Relationship,3,4
standard-sms.com-validXML-icsAccept-ERROR-!Invalid Insurance Hierarchy,3,4
standard-sms.com-validXML-icsAccept-ERROR-!Invalid PolicyHolder DOB,52,86
standard-sms.com-validXML-icsAccept-ERROR-!Missing POH Phone,33,54
standard-sms.com-validXML-icsAccept-ERROR-!Missing Relationship to Patient,4,7
standard-sms.com-validXML-icsAccept-ERROR-!SAX INV PROCVAL INSURANCE Pinadrnrph,3,4
standard-sms.com-validXML-icsAccept-ERROR-!Unexpected error creating Patient Insurance,10,15
standard-sms.com-validXML-icsAccept-ERROR-!Unmatched policy holder sex,4,6
standard-sms.com-validXML-icsReject-!Patient does not exist,1
standard-sms.com-validXML-icsReject-!Patient is On Hold,55
standard-providerTech-invalidXML-Missing Data-!Physician's phone number,83
standard-providerTech-invalidXML-Missing Data-!Contact's phone number,18
standard-providerTech-invalidXML-Invalid Data-!Physician's last name,318
standard-providerTech-invalidXML-Invalid Data-!Frequency of change,153
standard-providerTech-validXML-icsAccept-OK,343
standard-providerTech-validXML-icsAccept-ERROR-!Invalid Part,311,620
standard-providerTech-validXML-icsAccept-ERROR-!Agency is invalid and needs to be setup,11,23
standard-providerTech-validXML-icsAccept-ERROR-!Duplicate Line,1,2
standard-providerTech-validXML-icsAccept-ERROR-!EZ SHIP PATIENT,1,2
standard-providerTech-validXML-icsAccept-ERROR-!Insurance Duplicate Policy Number,2,4
standard-providerTech-validXML-icsAccept-ERROR-!Insurance Invalid Hierarchy primary,17,33
standard-providerTech-validXML-icsAccept-ERROR-!Insurance Missing,840,1672
standard-providerTech-validXML-icsAccept-ERROR-!Invalid INSURANCE Relationship,71,142
standard-providerTech-validXML-icsAccept-ERROR-!Invalid Part Unit of Measure,51,102
standard-providerTech-validXML-icsAccept-ERROR-!Invalid crossref for insurance name,99,196
standard-providerTech-validXML-icsAccept-ERROR-!Invalid Insurance Hierarchy,45,90
standard-providerTech-validXML-icsAccept-ERROR-!Invalid Insurance Plan,269,536
standard-providerTech-validXML-icsAccept-ERROR-!Invalid Insurance Policy Expiration Date,2,4
standard-providerTech-validXML-icsAccept-ERROR-!Invalid Policy Holder Zip,3,6
standard-providerTech-validXML-icsAccept-ERROR-!Invalid PolicyHolder DOB,97,194
standard-providerTech-validXML-icsAccept-ERROR-!Invalid Value for UnitOfMeasure,1,2
standard-providerTech-validXML-icsAccept-ERROR-!Missing POH First Name,68,135
standard-providerTech-validXML-icsAccept-ERROR-!Missing POH Gender,79,157
standard-providerTech-validXML-icsAccept-ERROR-!Missing POH Last Name,68,135
standard-providerTech-validXML-icsAccept-ERROR-!Missing POH Phone,54,107
standard-providerTech-validXML-icsAccept-ERROR-!Missing Policy Number,145,289
standard-providerTech-validXML-icsAccept-ERROR-!SAX INV PROCVAL INSURANCE Pinadrnrph,1
standard-providerTech-validXML-icsAccept-ERROR-!Ship To Agency Invalid Address,1
standard-providerTech-validXML-icsAccept-ERROR-!Unexpected error creating Patient Insurance,7,14
standard-providerTech-validXML-icsAccept-ERROR-!Unmatched policy holder sex,19,37
standard-providerTech-validXML-icsReject-!Order Contact Phone Number Invalid,5
standard-providerTech-validXML-icsReject-!Pat IT TECH,1
standard-providerTech-validXML-icsReject-!Patient DOB Invalid Validation,74
standard-providerTech-validXML-icsReject-!Patient does not exist,52
standard-providerTech-validXML-icsReject-!Patient First Name Invalid Validation,8
standard-providerTech-validXML-icsReject-!Patient Gender Invalid Validation,26
standard-providerTech-validXML-icsReject-!Patient is On Hold,72
standard-providerTech-validXML-icsReject-!Unknown,5
customized-invalidXML-Missing Data-!E1 Part number,4
customized-invalidXML-Invalid Data-!External part description,27
customized-invalidXML-Invalid Data-!Insurance name,12
customized-invalidXML-Invalid Data-!Wound location,11
customized-invalidXML-Invalid Data-!Type,10
customized-invalidXML-Invalid Data-!Physician phone number,8
customized-validXML-icsAccept-OK,128
customized-validXML-icsAccept-ERROR-!Invalid Part,164,434
customized-validXML-icsAccept-ERROR-!EZ SHIP PATIENT,1
customized-validXML-icsAccept-ERROR-!Insurance Invalid Hierarchy primary,1
customized-validXML-icsAccept-ERROR-!Insurance Missing,307,818
customized-validXML-icsAccept-ERROR-!Invalid INSURANCE Relationship,73,194
customized-validXML-icsAccept-ERROR-!Invalid crossref for insurance name,7,18
customized-validXML-icsAccept-ERROR-!Invalid Insurance Hierarchy,7,18
customized-validXML-icsAccept-ERROR-!Invalid Order Contact Phone,108,287
customized-validXML-icsAccept-ERROR-!Missing POH Date of Birth,71,190
customized-validXML-icsAccept-ERROR-!Missing POH First Name,71.190
customized-validXML-icsAccept-ERROR-!Missing POH Gender,76,201
customized-validXML-icsAccept-ERROR-!Missing POH Last Name,71,190
customized-validXML-icsReject-!Invalid Insurance Account,1
customized-validXML-icsReject-!Order Contact Phone Number Invalid,28
customized-validXML-icsReject-!Patient is On Hold,22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment