Skip to content

Instantly share code, notes, and snippets.

@Kcnarf

Kcnarf/.block

Last active December 3, 2018 08:34
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Kcnarf/1e6da47724c39156adb3 to your computer and use it in GitHub Desktop.
Save Kcnarf/1e6da47724c39156adb3 to your computer and use it in GitHub Desktop.
Download ZIP
timeline - correlation
Raw
.block
license: mit
Raw
README.md

An example of how to decide if 2 time series are correlated or not.

Graphically speaking, one can estimate the correlation of 2 time series by using a scatter plot of those 2 time series (right top graph): the more the points are aligned in a straight line, the more the time series are correlated.

Computationnaly speaking, the coefficient of correlation gives an insight of the dispersion of those points: the more the coefficient of correlation is near 1 (or -1), the more the scatter points are aligned, and the more the 2 time series are correlated.

Usages :

  • in the left graph, Drag & Drop a point to see the impact on the coefficient of correlation
  • in the left graph, Drag & Drop a timeline (to change each values of the related time serie), and see that it has no impact on the coefficient of correlation;
  • in the left graph, correlate or decorrelate time serie 2 and time serie 1 by introducing some random values; then see the impact on the coefficient of correlation;
  • in the left graph, increase or decrease the trend of the second time serie, and see that it has no impact on the coefficient of correlation;
  • in the left graph, inverse a trend, and see that it inverses the coefficient of correlation; when the 2 trends increases/decreases in the same way, the coefficient of correlation is positive; at the opposite, when one trend is increasing and the other is decreasing, the coefficient of correlation is negative;
  • in the left graph, disperse/concentrate the second time serie, and see that the more disperse is the second timeserie, the less it correlates with the first time serie;

Notes:

  • trend line computed using least square method

Acknowledgments:

Raw
index.html
<!DOCTYPE html>
<meta charset="utf-8">
<style>
body {
position: relative;
background-color: #ddd;
margin: auto;
}
#under-construction {
display: none;
position: absolute;
top: 200px;
left: 300px;
font-size: 40px;
}
#controls {
position: absolute;
top: 395px;
right: 10px;
font: 11px arial;
text-align: right;
}
.viz {
position: absolute;
background-color: white;
border-radius: 10px;
}
.viz#timelines {
top: 5px;
left: 5px;
}
.viz#correlation-plot {
top: 5px;
right: 5px;
}
.viz#correlation {
top: 385px;
right: 5px;
}
.flow {
position: absolute;
font-size: 30px;
color: darkgrey;
}
.flow#flow1 {
top: 310px;
left: 470px;
}
.flow#flow2 {
top: 350px;
right: 435px;
}
.axis path,
.axis line {
fill: none;
stroke: black;
shape-rendering: crispEdges;
}
.axis text {
font-family: sans-serif;
font-size: 11px;
}
#correlation-plot .axis.x text {
fill: steelblue;
}
#correlation-plot .axis.y text {
fill: seagreen;
}
.grid>line, .grid>.intersect {
fill: none;
stroke: #ddd;
shape-rendering: crispEdges;
vector-effect: non-scaling-stroke;
}
.legend {
font-size: 12px;
}
.dot {
fill: steelblue;
stroke: white;
stroke-width: 3px;
}
.dot.serie2 {
fill: seagreen;
}
.dot.correlated {
fill: grey;
stroke: none;
opacity: 0.5;
}
.dot.draggable:hover, .dot.dragging {
fill: pink;
cursor: ns-resize;
}
.timeline {
fill: none;
stroke: steelblue;
stroke-width: 2px;
opacity: 0.2;
}
.timeline.serie2 {
stroke: darkseagreen;
}
.timeline.draggable:hover, .timeline.dragging {
stroke: pink;
opacity: 1;
cursor: ns-resize;
}
.trend {
stroke: steelblue;
stroke-width: 2px;
}
.trend.serie2 {
stroke: green;
}
.trend.correlated {
stroke: grey;
}
#correlation-bar {
fill: grey;
}
</style>
<body>
<div id="timelines" class="viz">
<div id="controls">
<a href="#" onclick="correlate();">correlate</a> / <a href="#" onclick="decorrelate();">decorrelate</a> time serie 1 and time serie 2
<br/>
<br/>
time serie 1 : <a href="#" onclick="increaseTrend('serie1');">increase</a> / <a href="#" onclick="decreaseTrend('serie1');">decrease</a> / <a href="#" onclick="inverseTrend('serie1');">inverse</a> trend
<br/>
<a href="#" onclick="disperse('serie1');">increase</a> / <a href="#" onclick="concentrate('serie1');">decrease</a> dispersion
<br/>
<br/>
time serie 2 : <a href="#" onclick="increaseTrend('serie2');">increase</a> / <a href="#" onclick="decreaseTrend('serie2');">decrease</a> / <a href="#" onclick="inverseTrend('serie2');">inverse</a> trend
<br/>
<a href="#" onclick="disperse('serie2');">increase</a> / <a href="#" onclick="concentrate('serie2');">decrease</a> dispersion
<br/>
</div>
</div>
<div id="correlation-plot" class="viz"></div>
<div id="correlation" class="viz"></div>
<div id="flow1" class="flow">&#8614;</div>
<div id="flow2" class="flow">&#8615;</div>
<div id="under-construction">
UNDER CONSTRUCTION
</div>
<script src="https://d3js.org/d3.v3.min.js"></script>
<script>
var timeSerie = [];
var sameTrend = false;
var WITH_TRANSITION = true;
var WITHOUT_TRANSITION = false
var duration = 500;
var timelineVizDimension = {width: 960/2, height:500},
correlationPlotVizDimension = {width: 960/2, height:370},
correlationVizDimension = {width: 960/2, height:130},
vizMargin = 5,
flowWidth = 20
legendHeight = 20,
xAxisLabelHeight = 10,
yAxisLabelWidth = 10,
margin = {top: 20, right: 20, bottom: 20, left: 20},
timelineSvgWidth = timelineVizDimension.width - 2*vizMargin - flowWidth/2,
timelineSvgHeight = timelineVizDimension.height - 2*vizMargin,
correlationPlotSvgWidth = correlationPlotVizDimension.width - 2*vizMargin - flowWidth/2,
correlationPlotSvgHeight = correlationPlotVizDimension.height - 2*vizMargin - flowWidth/2,
correlationSvgWidth = correlationVizDimension.width - 2*vizMargin - flowWidth/2,
correlationSvgHeight = correlationVizDimension.height - 2*vizMargin - flowWidth/2,
timelineWidth = timelineSvgWidth - margin.left - margin.right - yAxisLabelWidth,
// timelineHeight = timelineSvgHeight - margin.top - margin.bottom - xAxisLabelHeight;
correlationPlotWidth = correlationPlotSvgWidth - margin.left - margin.right - yAxisLabelWidth,
correlationPlotHeight = correlationPlotSvgHeight - margin.top - margin.bottom - xAxisLabelHeight,
timelineHeight = correlationPlotHeight,
correlationWidth = correlationSvgWidth - margin.left - margin.right,
correlationHeight = correlationSvgHeight - margin.top - margin.bottom - xAxisLabelHeight - 2*legendHeight;
var drag1 = d3.behavior.drag()
.origin(function(d) { return d; })
.on("dragstart", dragStarted)
.on("drag", dragged1)
.on("dragend", dragEnded);
var drag2 = d3.behavior.drag()
.origin(function(d) { return d; })
.on("dragstart", dragStarted)
.on("drag", dragged2)
.on("dragend", dragEnded);
var dragTimeline1= d3.behavior.drag()
.origin(function(d) { return d; })
.on("dragstart", dragStarted)
.on("drag", draggedTimeline1)
.on("dragend", dragEnded);
var dragTimeline2= d3.behavior.drag()
.origin(function(d) { return d; })
.on("dragstart", dragStarted)
.on("drag", draggedTimeline2)
.on("dragend", dragEnded);
var x = d3.scale.linear()
.domain([0, 20])
.range([0, timelineWidth]);
var y = d3.scale.linear()
.domain([0, 50])
.range([0, -timelineHeight]);
var xPlot = d3.scale.linear()
.domain([0, 50])
.range([0, correlationPlotWidth]);
var yPlot = d3.scale.linear()
.domain([0, 50])
.range([0, -correlationPlotHeight]);
var xCorrelation = d3.scale.linear()
.domain([-1, 1])
.range([0, correlationWidth]);
var xAxisDef = d3.svg.axis()
.scale(x)
.ticks(11);
var yAxisDef = d3.svg.axis()
.scale(y)
.orient("left");
var xAxisPlotDef = d3.svg.axis()
.scale(xPlot);
var yAxisPlotDef = d3.svg.axis()
.scale(yPlot)
.orient("left");
var xAxisCorrelationDef = d3.svg.axis()
.scale(xCorrelation)
.ticks(5);
var svg = d3.select("#timelines").append("svg")
.attr("width", timelineSvgWidth)
.attr("height", timelineSvgHeight)
.append("g")
.attr("transform", "translate(" + [margin.left, margin.top] + ")");
var container = svg.append("g")
.attr("id", "graph")
.attr("transform", "translate(" + [yAxisLabelWidth, timelineHeight] + ")");
var grid = container.append("g")
.attr("class", "grid");
var intersects = [];
d3.range(2, x.invert(timelineWidth), 2).forEach(function(a) { d3.range(5, y.invert(-timelineHeight),5).forEach(function(b) { intersects.push([a,b])})});
grid.selectAll(".intersect")
.data(intersects)
.enter().append("path")
.classed("intersect", true)
.attr("d", function(d) { return "M"+[x(d[0])-1,y(d[1])]+"h3M"+[x(d[0]),y(d[1])-1]+"v3"});
container.append("text")
.attr("transform", "translate(" + [timelineWidth/2, -timelineHeight] + ")")
.attr("text-anchor", "middle")
.text("Timelines");
container.append("g")
.attr("class", "axis x")
.call(xAxisDef)
.append("text")
.attr("x", timelineWidth)
.attr("y", -6)
.style("text-anchor", "end")
.text("Time");
container.append("g")
.attr("class", "axis y")
.call(yAxisDef)
.append("text")
.attr("transform", "rotate(-90)")
.attr("x", timelineHeight)
.attr("y", 16)
.style("text-anchor", "end")
.text("Amount");
var timeline1 = container.append("path")
.datum(1)
.classed("timeline serie1 draggable", true)
.attr("d", line1)
.call(dragTimeline1);
var timeline2 = container.append("path")
.datum(2)
.classed("timeline serie2 draggable", true)
.attr("d", line2)
.call(dragTimeline2);
var dotContainer = container.append("g")
.classed("dots", true);
svg = d3.select("#correlation-plot").append("svg")
.attr("width", correlationPlotSvgWidth)
.attr("height", correlationPlotSvgHeight)
.append("g")
.attr("transform", "translate(" + [margin.left, margin.top] + ")");
var container2 = svg.append("g")
.attr("id", "graph correlated")
.attr("transform", "translate(" + [yAxisLabelWidth, correlationPlotHeight] + ")");
var grid2 = container2.append("g")
.attr("class", "grid");
var intersects = [];
d3.range(5, xPlot.invert(correlationPlotWidth), 5).forEach(function(a) { d3.range(5, yPlot.invert(-correlationPlotHeight),5).forEach(function(b) { intersects.push([a,b])})});
grid2.selectAll(".intersect")
.data(intersects)
.enter().append("path")
.classed("intersect", true)
.attr("d", function(d) { return "M"+[xPlot(d[0])-1,yPlot(d[1])]+"h3M"+[xPlot(d[0]),yPlot(d[1])-1]+"v3"});
container2.append("text")
.attr("transform", "translate(" + [correlationPlotWidth/2, -correlationPlotHeight] + ")")
.attr("text-anchor", "middle")
.text("Scatter plot");
container2.append("text")
.attr("transform", "translate(" + [correlationPlotWidth/2, -correlationPlotHeight+15] + ")")
.attr("text-anchor", "middle")
.style("font-size", "12px")
.text("between time serie 1 and time serie 2");
container2.append("g")
.attr("class", "axis x")
.call(xAxisPlotDef)
.append("text")
.attr("x", correlationPlotWidth)
.attr("y", -6)
.style("text-anchor", "end")
.text("Amount (from 1st time serie)");
container2.append("g")
.attr("class", "axis y")
.call(yAxisPlotDef)
.append("text")
.attr("transform", "rotate(-90)")
.attr("x", correlationPlotHeight)
.attr("y", 16)
.style("text-anchor", "end")
.text("Amount (from 2nd time serie)");
var correlatedDotContainer = container2.append("g")
.classed("dots correlated", true);
var correlatedTrendLine = container2.append("line")
.attr("class", "trend correlated")
.attr("x1", xPlot(0))
.attr("y1", yPlot(0))
.attr("x2", xPlot(50))
.attr("y2", yPlot(50));
container = d3.select("#correlation").append("svg")
.attr("width", correlationSvgWidth)
.attr("height", correlationSvgHeight)
.append("g")
.attr("transform", "translate(" + [margin.left, margin.top] + ")");
var graph3 = container.append("g")
.attr("id", "graph correlation")
.attr("transform", "translate(" + [0, correlationHeight] + ")");
graph3.append("text")
.attr("transform", "translate(" + [correlationWidth/2, -correlationHeight] + ")")
.attr("text-anchor", "middle")
.text("Coefficient of correlation");
graph3.append("g")
.attr("class", "axis x")
.call(xAxisCorrelationDef);
var legend3 = container.append("g")
.classed("legend", true)
.attr("transform", "translate(" + [0, correlationHeight+xAxisLabelHeight] + ")");
legend3.append("g")
.selectAll("text")
.data(["Perfect", "Inverted", "Correlation"])
.enter().append("text")
.attr("x", 0)
.attr("y", function(d,i) { return 25 + i*10; })
.style("text-anchor", "start")
.text( function(d) { return d; });
legend3.append("g")
.attr("transform", "translate(" + [correlationWidth/2, 0] + ")")
.selectAll("text")
.data(["No", "Correlation"])
.enter().append("text")
.attr("x", 0)
.attr("y", function(d,i) { return 25 + i*10; })
.style("text-anchor", "middle")
.text( function(d) { return d; });
legend3.append("g")
.attr("transform", "translate(" + [correlationWidth, 0] + ")")
.selectAll("text")
.data(["Perfect", "Correlation"])
.enter().append("text")
.attr("x", 0)
.attr("y", function(d,i) { return 25 + i*10; })
.style("text-anchor", "end")
.text( function(d) { return d; });
var correlationBar = graph3.append("path")
.attr("id", "correlation-bar")
.attr("d", "M"+[xCorrelation(0), -1]+"v-10H"+xCorrelation(0.34)+"v10z")
d3.csv("timeserie.csv", dottype, function(error, dots) {
updateDots(WITHOUT_TRANSITION);
updateCorrelatedDots(WITHOUT_TRANSITION);
updateTimelines(WITHOUT_TRANSITION);
updateTrends(WITHOUT_TRANSITION);
});
function dottype(d) {
d.x = +d.x;
d.y1 = +d.y+d.x/2-10;
d.y2 = +d.y1/2+(10*Math.random()-1);
timeSerie.push(d);
return d;
}
var line1 = d3.svg.line()
.x(function(d) { return x(d.x); })
.y(function(d) { return y(d.y1); });
var line2 = d3.svg.line()
.x(function(d) { return x(d.x); })
.y(function(d) { return y(d.y2); });
function changeTrend(serieName, scale) {
var y = (serieName==="serie1")? "y1" : "y2";
var serieLength = timeSerie.length;
var ySum = 0;
timeSerie.forEach(function(d){
ySum += d[y];
});
var mean = ySum/serieLength;
timeSerie.forEach(function(d){
d[y] = mean + scale*(d[y]-mean);
});
updateDots(WITH_TRANSITION);
updateTimelines(WITH_TRANSITION);
updateCorrelatedDots(WITH_TRANSITION);
updateTrends(WITH_TRANSITION);
}
function inverseTrend(serieName) {
changeTrend(serieName, -1)
}
function increaseTrend(serieName) {
changeTrend(serieName, 1.6)
}
function decreaseTrend(serieName) {
changeTrend(serieName, 0.625)
}
function changeDispersion(serieName, scale) {
var y = (serieName==="serie1")? "y1" : "y2";
var serieLength = timeSerie.length;
var timeInterval = 1;
var ySum = 0;
var timeYSum = 0;
timeSerie.forEach(function(d){
ySum += d[y];
timeYSum += d.x*d[y];
});
var trend = (12*timeYSum - 6*(serieLength+1)*ySum)/(timeInterval*serieLength*(serieLength-1)*(serieLength+1));
var intercept = (2*(2*serieLength+1)*ySum - 6*timeYSum)/(serieLength*(serieLength-1));
var expected;
timeSerie.forEach(function(d){
expected = d.x*trend + intercept;
d[y] = expected + scale*(d[y]-expected);
});
updateDots(WITH_TRANSITION);
updateTimelines(WITH_TRANSITION);
updateCorrelatedDots(WITH_TRANSITION);
updateTrends(WITH_TRANSITION);
}
function disperse(serieName) {
changeDispersion(serieName, 1.6)
}
function concentrate(serieName) {
changeDispersion(serieName, 0.625)
}
function correlate() {
changeCorrelation(0);
updateDots(WITH_TRANSITION);
updateTimelines(WITH_TRANSITION);
updateCorrelatedDots(WITH_TRANSITION);
updateTrends(WITH_TRANSITION);
}
function decorrelate() {
changeCorrelation(10);
updateDots(WITH_TRANSITION);
updateTimelines(WITH_TRANSITION);
updateCorrelatedDots(WITH_TRANSITION);
updateTrends(WITH_TRANSITION);
}
function changeCorrelation(kindOfDispersion) {
var intercept = (5 + 15*Math.random())*((kindOfDispersion)/10);
var trend = 0.5*((kindOfDispersion)/10);
var couplingCoefficient = Math.random() * 1.7 * ((10-kindOfDispersion)/10);
timeSerie.forEach( function(d, i) {
d.y2 = intercept+trend*i + d.y1*couplingCoefficient+kindOfDispersion*(Math.random()-0.5);
});
}
function updateDots(withTransition) {
var dots = dotContainer.selectAll(".dot.serie1")
.data(timeSerie);
dots.enter()
.append("circle")
.classed("dot draggable serie1", true)
.attr("r", 5)
.attr("cx", function(d) { return x(d.x); })
.attr("cy", function(d) { return y(d.y1); })
.call(drag1);
dots.transition()
.duration(withTransition? duration : 0)
.attr("cy", function(d) { return y(d.y1); })
dots = dotContainer.selectAll(".dot.serie2")
.data(timeSerie);
dots.enter()
.append("circle")
.classed("dot draggable serie2", true)
.attr("r", 5)
.attr("cx", function(d) { return x(d.x); })
.attr("cy", function(d) { return y(d.y2); })
.call(drag2);
dots.transition()
.duration(withTransition? duration : 0)
.attr("cy", function(d) { return y(d.y2); })
}
function updateTimelines(withTransition) {
timeline1.data(timeSerie).transition()
.duration(withTransition? duration : 0)
.attr("d", line1(timeSerie));
timeline2.data(timeSerie).transition()
.duration(withTransition? duration : 0)
.attr("d", line2(timeSerie));
}
function updateCorrelatedDots(withTransition) {
var correlatedDots = correlatedDotContainer.selectAll(".dot.correlated")
.data(timeSerie);
correlatedDots.enter()
.append("circle")
.classed("dot correlated", true)
.attr("r", 3.5)
.attr("cx", function(d) { return xPlot(d.y1); })
.attr("cy", function(d) { return yPlot(d.y2); });
correlatedDots.transition()
.duration(withTransition? duration : 0)
.attr("cx", function(d) { return xPlot(d.y1); })
.attr("cy", function(d) { return yPlot(d.y2); })
}
function updateTrends(withTransition) {
var serieLength = timeSerie.length;
var timeInterval = 1
var y1Sum = 0;
var squareY1Sum = 0;
var y2Sum = 0;
var squareY2Sum = 0;
// below sums are for trend lines and correlation
var y1Y2Sum = 0;
timeSerie.forEach(function(d){
y1Sum += d.y1;
squareY1Sum += Math.pow(d.y1, 2);
y2Sum += d.y2;
squareY2Sum += Math.pow(d.y2, 2);
y1Y2Sum += (d.y1)*(d.y2);
});
var y1Mean = y1Sum/serieLength;
var y2Mean = y2Sum/serieLength;
var y1Variance = squareY1Sum/serieLength - Math.pow(y1Mean, 2);
var y2Variance = squareY2Sum/serieLength - Math.pow(y2Mean, 2);
var y1StdDev = Math.pow(y1Variance, 0.5)
var y2StdDev = Math.pow(y2Variance, 0.5)
var y1Y2Covariance = y1Y2Sum/serieLength - y1Mean*y2Mean;
var correlatedTrend = y1Y2Covariance/(y1Variance);
var correlatedIntercept = y2Mean - correlatedTrend*y1Mean;
var correleationCoefficient = y1Y2Covariance/(y1StdDev*y2StdDev);
correlatedTrendLine
.transition()
.duration(withTransition? duration : 0)
.attr("y1", yPlot(correlatedIntercept))
.attr("y2", yPlot(correlatedTrend*50+correlatedIntercept));
correlationBar
.transition()
.duration(withTransition? duration : 0)
.attr("d", "M"+[xCorrelation(0), -1]+"v-10H"+xCorrelation(correleationCoefficient)+"v10z")
}
function dragStarted(d) {
d3.select(this).classed("dragging", true);
}
function dragged1(d) {
d.y1 += y.invert(d3.event.dy)
updateDots(WITHOUT_TRANSITION);
updateTimelines(WITHOUT_TRANSITION);
updateCorrelatedDots(WITHOUT_TRANSITION);
updateTrends(WITHOUT_TRANSITION);
}
function dragged2(d) {
d.y2 += y.invert(d3.event.dy)
updateDots(WITHOUT_TRANSITION);
updateTimelines(WITHOUT_TRANSITION);
updateCorrelatedDots(WITHOUT_TRANSITION);
updateTrends(WITHOUT_TRANSITION);
}
function dragEnded(d) {
d3.select(this).classed("dragging", false);
}
function draggedTimeline1(d) {
var rawdy = y.invert(d3.event.dy);
timeSerie.forEach(function(d){
d.y1 += rawdy;
});
updateDots(WITHOUT_TRANSITION);
updateTimelines(WITHOUT_TRANSITION);
updateCorrelatedDots(WITHOUT_TRANSITION);
updateTrends(WITHOUT_TRANSITION);
}
function draggedTimeline2(d) {
var rawdy = y.invert(d3.event.dy);
timeSerie.forEach(function(d){
d.y2 += rawdy;
});
updateDots(WITHOUT_TRANSITION);
updateTimelines(WITHOUT_TRANSITION);
updateCorrelatedDots(WITHOUT_TRANSITION);
updateTrends(WITHOUT_TRANSITION);
}
</script>
Raw
timeserie.csv
x y
1 24
2 25
3 22
4 28
5 26
6 29
7 31
8 30
9 34
10 34
11 33
12 34
13 39
14 44
15 42
16 44
17 41
18 41
19 44
20 44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment