Add "daily" option to bugdaystats.py

Recently, we are triaging bugs of Tempest on the launchpad and that
work was fine to clean up old bugs and fixed bugs with different
patches.
Current bugdaystats shows it in last several days statically, and it
would be nice to add more long terms like in 30 days or a release to see
bug triage trends.

This patch adds a "daily" option to bugdaystats.py for making daily data
json files. The file can be used to show long term data in other graphs.

Change-Id: Ieac396389ce13468b717d03264164c77f7c36104

README.rst

@@ -19,7 +19,11 @@ python bugdaystats.py output
 'output' is the name of the directory you will generate data
 and HTML files to (if they don't exist yet). It should contain a
 'js' subdirectory containing JavaScript include files, but
-otherwise be empty.
+otherwise be empty. In addition, it can contain two types of data
+files. One file contains data in each time when running the script,
+and another one is created daily. That means the creation is skipped
+if the previous data is in the same day. The data file is useful for
+showing long-term bug situation.
 
 You'll need to run the script at least twice to generate enough
 stats to get a graph.
@@ -39,3 +43,8 @@ page.
 You can also optionally specify a 'rotation' parameter. Entries older
 than the value (in days) will be removed from the dataset, resulting
 in a rolling view of bug activity.
+
+And you can also optionally specify a 'daily' parameter to enable the
+feature of daily data collection and showing its graphs.
+'daily_rotation' parameter is for daily data collection feature but
+it is same as 'rotation' parameter.

bugdaystats.py

@@ -41,20 +41,29 @@ def create_files(templatepath, outputpath, projects):
             template.stream(project=project).dump(projectfile)
 
 
-def update_stats(outputpath, project_name, rotation):
+def update_stats(outputpath, project_name, rotation, daily=False):
 
     now = int(time.time())
     records = []
     counts = {}
     project = launchpad.projects[project_name]
-    project_stats_filename = os.path.join(outputpath,
+    if daily:
-                                          "%s-bug-stats.json" % (project_name))
+        filename = "%s-bug-stats-daily.json" % project_name
+    else:
+        filename = "%s-bug-stats.json" % project_name
+    project_stats_filename = os.path.join(outputpath, filename)
 
     try:
         data_file = open(project_stats_filename, 'r')
         json_data = json.load(data_file)
         data_file.close()
         for record in json_data['records']:
+            if daily:
+                if (now - record['date']) < (24 * 60 * 60):
+                    # Skip to update stats if the recode contains the same
+                    # day data.
+                    return
+
             if rotation:
                 if (now - record['date']) > (rotation * 24 * 60 * 60):
                     continue
@@ -159,6 +168,8 @@ if __name__ == '__main__':
         config = json.load(configfile)
     projects = config['projects']
     rotation = config.get('rotation')
+    daily = config.get('daily')
+    daily_rotation = config.get('daily_rotation')
     openstack_status = config.get('openstack_status')
 
     # Create files in output directory, if needed
@@ -170,3 +181,6 @@ if __name__ == '__main__':
 
     for p in projects:
         update_stats(outputpath, p['project'], rotation)
+
+        if (daily):
+            update_stats(outputpath, p['project'], daily_rotation, daily=True)

config.js.sample

@@ -10,5 +10,7 @@
         { "project": "openstack-manuals", "title": "Manuals" },
         { "project": "tempest" }
     ],
-    "rotation": 2
+    "rotation": 2,
+    "daily": true,
+    "daily_rotation": 183
 }