LBRY Block Explorer

LBRY Claims • macos-development-01-performance-testing

c3449fdd70a386549f9f8519008eddf5e560d9ec

Published By
Created On
1 May 2020 17:03:49 UTC
Transaction ID
Cost
Safe for Work
Free
Yes
macOS Development 01 Performance Testing
# Performance Testing

## Introduction

This video is going to document the process of performance testing production code that is being refactored before being used in a couple of app updates. The video will be entirely in Xcode, because the code is eventually going to be integrated into production apps.

The video is aimed at anybody who has an interest in the process of macOS and iOS development. Some knowledge of programming basics might be useful but not entirely necessary. The video is going to skim over a lot of topics and is intended more as an insight rather than a tutorial.

## Moving from NSData to Data

In a couple of production apps I have been using my own Swift wrapper around the CommonCrypto C api. Along with the CommonCrypto wrapper I was also using an NSData extension to get a Hex string representation of the digest.

CryptoKit, which is a native `Swift` api for common cryptographic operations, has recently been introduced and I am going to replace the old CommonCrypto wrapper code on a couple of production apps. CryptoKit has the advantage of being a native Swift api and will allow me to simplify my wrapper code and remove the C bridging header altogether. At the same time I will reimplement the NSData extension as a native Swift `Data` extension and take the time to look at the implementation and see if there are any performance gains to be had.

So, the exercise today is to ensure that my new `Data` implementation is correct and is roughly comparable if not better in terms of performance than the original implementation.

## Setup

I have included my original `NSData` hex extension along with the intended `Data` replacement. There are three different implementations which are to be tested.

To provide some fairly realistic workloads I have included `TestData`. This is just wraps up in a struct a valid SHA256 and SHA512 along with a geojson example and keeps it out of the `main.swift`.

I have also included the `Performance` struct and static functions which will run the test code plenty of times and abstract away the averaging of the results to hopefully smooth out the impact other system resources have on the running code. The `Performance.measure` static method can take a couple of parameters to control how it works, but the basics of it are a number of iterations which is how many times to execute the test code over each run. Executing the code multiple times for each run means that code that executes very fast can still be captured. The purpose of this method is to compare performance, rather than measure absolute execution time. This is then run multiple times so that we can accumulate minimum and maximum values, calculating a mean execution time at the end. The deviation between min and max will just give an indication of how stable the results are.

## Stage 1 - Correctness

The first step is to confirm that the `Data` extension produces the expected output. In the production code I have unit tests that do this more extensively, but just to make sure changes that I make during this optimisation don't break the implementation, a simple equality check will be performed.

```swift
let digest = TestData.sha256
let digestData = Data(hexString: digest)!

let hexLoop = digestData.hexLoop
let hexReduce = digestData.hexReduce
let hexReduceAppend = digestData.hexReduceAppend

let nsdata = NSData(data: digestData)
let hexNSData = nsdata.hex

print(digest == hexLoop && digest == hexReduce && digest == hexNSData && digest == hexReduceAppend)
// true
```

This snipped creates a `Data` instance and runs the new extension methods along with my original NSData method and compares them all. We expect the console to print `true`.

It is now time to measure the performance of the methods.

## Stage 2 - Performance

```swift
[
Performance.measure(label: "Loop") { let _ = digestData.hexLoop },
Performance.measure(label: "ReduceInto") { let _ = digestData.hexReduceInto },
Performance.measure(label: "Reduce") { let _ = digestData.hexReduce },
Performance.measure(label: "NSData") { let _ = nsdata.hex }
]
.sorted { $0.mean < $1.mean }
.forEach { print($0) }
```

A sample output on a 2018 MacBook Pro shows that all the results are in the same ball park. With `hexReduceInto` having a slight edge over the other `Data` methods and within a cats whisker of the original `NSData` implementation. In fact the overall, best case, winner is the `hexReduceInto`. I find this interesting as I had expected the for loop version to be slightly more performant, demonstrating the need for this exercise!

```sh
true
PerformanceResult(label: "ReduceInto", min: 0.08666801452636719, max: 0.089790940284729, mean: 0.08822681903839111, dev: 0.0031229257583618164)
PerformanceResult(label: "NSData", min: 0.08542799949645996, max: 0.09727001190185547, mean: 0.08874142169952393, dev: 0.011842012405395508)
PerformanceResult(label: "Loop", min: 0.08816790580749512, max: 0.09841704368591309, mean: 0.09236581325531006, dev: 0.010249137878417969)
PerformanceResult(label: "Reduce", min: 0.10085892677307129, max: 0.10363304615020752, mean: 0.10235416889190674, dev: 0.0027741193771362305)
```

## Stage 3 - Analysis

> I'm going to be using forced unwrapped optionals here. This is not something I would recommend to do in production, but I am doing it here because I have precise control over the data to be operated on and a crash in this test is no problem

To make it a little clearer which was the winner a little bit of printing to the console will help:

```swift
let winner = results[0]
let secondPlace = results[1]

print("\nThe winner is",
winner.label,
"averaging",
Performance.ops(executionTime: winner.mean, executionCount: executionCount),
"ops / s")

print(winner.label,
"best effort was",
Performance.ops(executionTime: winner.min, executionCount: executionCount),
"ops / s")

print("\nSecond place goes to",
secondPlace.label,
"averaging",
Performance.ops(executionTime: secondPlace.mean, executionCount: executionCount),
"ops / s")
```

## Conculsion

As we can see over numerous runs the flaws in the testing strategy start to show up and overwhelm the results, which are very close. Swapping from NSData to Data has a couple of benefits and the `reduceInto` implementation is, in my opinion, considerably more readable with comparable performance.


Author
Content Type
Unspecified
video/mp4
Language
English
Open in LBRY

More from the publisher

Controlling
VIDEO
FINDI
Controlling
VIDEO
WRITI
Controlling
VIDEO
USING
Controlling
VIDEO
NOD
Controlling
VIDEO
MACOS
Controlling
VIDEO
MOTIV
Controlling
VIDEO
NOD 0
Controlling
VIDEO
NOD 0
Controlling
VIDEO
MACOS