======================
Mining Python Software
======================

* Sarah Mount
* http://www.slideshare.net/snim2/mining-pythonsoftwarepyconuk13

What do we know about software?
===============================

* How to make it correct
* How long it will take to write
* expected bugs per kloc

Er ... yeah

Lots of studies done on students (as they're available to academics) but not necessarily good for industry.

WARNING: this is a work in progress - interim numbers

Where to find info
==================

Don't write your own scripts to scrape github, google code etc.  Use 

* ohloh - good API - github, google code, bitbucket ...
* github archive
* google bigquery interface

Code Clones
===========

* type 1: identical code, copy and pasted
* type 2: identical code, names changed, layout, comments etc
* type 3: as type 2, plus further modifications such as change in statement
* type 4: completely different code, same semantics (no tools auto-detect this yet)

Tool: Clone Digger

Take 500 most forked python repos on github, clone them, run clone digger (and other tools) over all code and collate results.

Code clones - most python projects similar - very low - but some stand out.  evervim and everpad have over 40,000 clones.

* Normalised clone density - take into account size of project
* length of code clones (lines of code cloned)

Aggregate Sentiment of Comments
===============================

Most pretty neutral - NewsBlur was amazingly happy