First some background. Around a year ago I joined an existing project at a new client. As they had no continuous integration or deployment, we added it and implemented the versions numbering scheme as described in Using Continuous Integration to version your application. We also displayed the version number of the site in the footer of the home page, which has helped in logging issues against specific versions.

As part of our improvement process, we started using Trello to manage our tickets in a kanban type flow. The tickets moved from Open => In Progress => Done, where done would be ready for testing with a specific version number. With the speed of our workflow (5 developers to 1 tester), we discovered that we needed to get more testing done by the developers. This helped to avoid siloing and freed up our tester to concentrate on more detailed issues.

Our new process was Open => In Progress => Dev Testing => User Testing. However this meant that there could be a combination of issues in the Test environment, some in Dev Testing and some in User Testing. We are not using Trello’s custom fields to track versions, just free text in the comments, so tracking down which environment the changes for a ticket has been pushed to, could be a challenge.

While the whole team was working, it was easy enough to watch the tickets (and with good communication) get an approximate idea of how far a code change had got in the deployment pipeline. But come August 2016 our product development manager and lead QA tester went on leave for two weeks and the dev team kept working. When he came back I couldn’t instantly tell him what changes where in which environment, only give him a version number!

We needed a better way to establish what commits where linked to which Trello tickets, what version they had been released in, and which environment the version had reached. We were already logging our commits with the Trello links, so my first step was to extract all the information from the git log and put it into a useful format:

git log --since=30.days --full

I wanted to display this in a browser so JSON seemed like an obvious choice, and luckily the pretty format is powerful.

git log -n 5 --pretty=format:"{ "hash": "%H", "subject": "%s", "commit_notes": "%N", "author": { "name": "%aN","email": "%aE","date": "%aD" }, },"

This produces lots of (invalid) JSON strings from your log, and you still need to include the opening and closing {} braces to make it a valid JSON file. Plus it does not handle if you have line breaks in your commit messages. Or tab breaks. Or quotes. Or any other characters that need to be escaped. And I want to run this from the build server. Err… Ok PowerShell it is! The full script is below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
<#
Generate the change log in json
#>
param(
    [ValidateSet("True","False", 0, 1)]
    [ValidateNotNullOrEmpty()]
    [string]$OutputAsString = "False",
    [string]$OutputFilename = "changelog.json",
    [string]$GitDirectory = ""
)

# Get Start Time
$startDTM = (Get-Date)

# required to generate a proper exit code on errors.
trap 
{ 
  Write-Error $_.Exception | format-list -force
  exit 1 
}

function ConvertStringToBoolean ([string]$value) {
	$value = $value.ToLower();

	switch ($value) {
		"true" { return $true; }
		"1" { return $true; }
		"false" { return $false; }
		"0" { return $false; }
	}
}

Write-Host ("OutputFilename: {0}"-f $OutputFilename)
Write-Host ("GitDirectory: {0}"-f $GitDirectory)
Write-Host ("OutputAsString: {0}"-f $OutputAsString)

[bool]$outputJsonAsString = ConvertStringToBoolean($OutputAsString);

Write-Host ("Removing file '{0}'..." -f $OutputFilename)

Remove-Item $OutputFilename -Force -ErrorAction Ignore

$currentDirectory = Get-Location

Write-Host ("Changing to folder '{0}'..." -f $GitDirectory)

Set-Location $GitDirectory

# Using ^@^ as an easy to find delimiter (and unlikey to be in our git logs)
git log --since=10.days --pretty=format:'{ ^@^shorthashId^@^: ^@^%h^@^, ^@^hash^@^: ^@^%H^@^,^@^parents^@^:^@^%P^@^,^@^tree^@^:^@^%T^@^,^@^author^@^: ^@^%an^@^,^@^date^@^: ^@^%cI^@^,^@^authordate^@^: ^@^%aI^@^,^@^subject^@^: ^@^%s^@^,^@^body^@^: ^@^%B^@^ },^^' > $OutputFilename

Write-Host ("Loading git output from file '{0}'..." -f $OutputFilename)

# use Out String to preserve the formatting
[string]$logContent = get-content($OutputFilename) | Out-String

if (![String]::IsNullOrEmpty($logContent)){

    Write-Host "Processing git log file..."

    # escape backslach in any body text
    $escapeBackslash = "\\"
    if ($outputJsonAsString -eq $true){
        $escapeSingleQuotes = "\\\"
    }
    $logContent = $logContent.replace("\", $escapeBackslash)

    # Remove the end of line between commit messages.
    $logContent = $logContent.replace("^^`r`n", "")

    # Change new line characters in the json
    $logContent = $logContent.replace("`r`n", "\\n")

    # Change the tab characters in the json
    $logContent = $logContent.replace("`t", "\\t")

    # escape quotes in any body text
    $escapeQuotes = '\"'
    if ($outputJsonAsString -eq $true){
        $escapeQuotes = '\\\"'
    }
    $logContent = $logContent.replace('"', $escapeQuotes)

    $escapeSingleQuotes = "\\'"
    if ($outputJsonAsString -eq $true){
        $escapeSingleQuotes = "\'"
    }
    $logContent = $logContent.replace("'", $escapeSingleQuotes)

    # Change string delimiters
    $quote = '"'
    if ($outputJsonAsString -eq $true){
        $quote = '\"'
    }
    $logContent = $logContent.replace('^@^', $quote)
    
    # remove trailing comma
    $logContent = $logContent.Substring(0, $logContent.Length - 1)
    
    #change into a json array
    $logContent = '[' + $logContent +  ']'
    
    if ($outputJsonAsString -eq $true){
        $logContent = '"' + $logContent +  '"'
    }

    Write-Host ("Writing json file '{0}'..." -f $OutputFilename)

    $logContent | out-file $OutputFilename

    $global:LASTEXITCODE = 0
}
else{
    Write-Error ("No git log export found!")
    $global:LASTEXITCODE = 1
}

# Get End Time
$endDTM = (Get-Date)

# Echo Time elapsed
Write-host "Elapsed Time taken: $(($endDTM-$startDTM).totalseconds) seconds"

Set-Location $currentDirectory

exit $global:LASTEXITCODE

Running this PowerShell script on the VSCode Editor git repository outputs the following JSON file (truncated):

[   {
		"shorthashId" : "196307b",
		"hash" : "196307bb846109c90e94738c1bf32f7146ce9f96",
		"parents" : "379ed09d000bbb50428dc5ceb3ad0a1b347b1391",
		"tree" : "2506fa2b7bd1cc40cad67438943a5a9fed237389",
		"author" : "Sandeep Somavarapu",
		"date" : "2017-02-03T12:16:58+01:00",
		"authordate" : "2017-02-03T12:16:58+01:00",
		"subject" : "Fix #19791",
		"body" : "Fix #19791\\n"
	}, {
		"shorthashId" : "379ed09",
		"hash" : "379ed09d000bbb50428dc5ceb3ad0a1b347b1391",
		"parents" : "1a813ef39de9a19d3686176bd0afe2f09a047479",
		"tree" : "39cd042c6b69e1ce3bcc6ecbda7c63ed293a38d6",
		"author" : "Dirk Baeumer",
		"date" : "2017-02-03T11:50:04+01:00",
		"authordate" : "2017-02-03T11:50:04+01:00",
		"subject" : "Merged in translations",
		"body" : "Merged in translations\\n"
	}, {
		"shorthashId" : "1a813ef",
		"hash" : "1a813ef39de9a19d3686176bd0afe2f09a047479",
		"parents" : "97f04b60a6dfc72c3fda2d4e63e225204436aa05",
		"tree" : "00a6c0901b0a06cb652a153ddd0a3dc81eccac66",
		"author" : "Benjamin Pasero",
		"date" : "2017-02-03T11:26:52+01:00",
		"authordate" : "2017-02-03T11:26:52+01:00",
		"subject" : "Menu Bar Resets to Visible When Reopening Window (fixes #19777)",
		"body" : "Menu Bar Resets to Visible When Reopening Window (fixes #19777)\\n"
	}, {
		"shorthashId" : "97f04b6",
		"hash" : "97f04b60a6dfc72c3fda2d4e63e225204436aa05",
		"parents" : "7a9470db31df52a9e3c6610f7814412efb059f6b",
		"tree" : "68cead60e31e10647c869359d2391aad86272e07",
		"author" : "Joao Moreno",
		"date" : "2017-02-03T11:21:28+01:00",
		"authordate" : "2017-02-03T11:21:28+01:00",
		"subject" : "weird backticks",
		"body" : "weird backticks\\n"
	}
]

Now we need to display this output in a browser. Part 2 can be found here.