Benchmarking JuliaImages.jl

Using:

BenchmarkTools.jl | jl

timeit | py

Against :

  1. OpenCV

Version used : 4.1.0

  1. scikit-image

Version used : 0.16.2

  1. (PIL using Pillow)[https://github.com/PseudoCodeNerd/codein-julia/blob/master/benchmark-openCV-JuliaImage/bench-3.ipynb]

Version used : 6.1.0

Machine on which Benchmarks were carried out :

64Bit Windows 10 with 16 GB of RAM and i5-7200U@2.5Ghz

Sample image from https://testimages.juliaimages.org/

Note : Compared mean times by timing only one sample in timeit.

#Getting the Required Packages
using Images, ImageTransformations, FileIO, BenchmarkTools, TestImages

Test 1: Loading Image 

@benchmark img = load("img/mountainstream.png")

BenchmarkTools.Trial: 
  memory estimate:  3.39 MiB
  allocs estimate:  322
  --------------
  minimum time:     76.631 ms (0.00% GC)
  median time:      105.579 ms (0.00% GC)
  mean time:        110.319 ms (0.41% GC)
  maximum time:     209.470 ms (0.00% GC)
  --------------
  samples:          46
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 110.319
OpenCV 25.222
PIL 0.248

$PIL < OpenCV < Images.jl$

Test 2: Saving Image 

img = load("img/mountainstream.png")
@benchmark save("copy_julia.png", img)

BenchmarkTools.Trial: 
  memory estimate:  3.39 MiB
  allocs estimate:  305
  --------------
  minimum time:     257.997 ms (0.00% GC)
  median time:      336.976 ms (0.00% GC)
  mean time:        316.100 ms (0.10% GC)
  maximum time:     368.297 ms (0.00% GC)
  --------------
  samples:          16
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 316.1
OpenCV 40.224
PIL 174.674

$OpenCV < PIL < Images.jl$

Test 3: Resizing Image 

img = load("img/mountainstream.png")
@benchmark big_img = imresize(img, ratio=5)

BenchmarkTools.Trial: 
  memory estimate:  29.25 MiB
  allocs estimate:  6
  --------------
  minimum time:     324.998 ms (0.00% GC)
  median time:      344.478 ms (0.08% GC)
  mean time:        376.705 ms (4.31% GC)
  maximum time:     583.193 ms (0.00% GC)
  --------------
  samples:          14
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 376.705
OpenCV 33.105
PIL 57.582

$ OpenCV < PIL < Images.jl$

Test 4: Greyscaling Image 

img = load("img/mountainstream.png")
@benchmark gray_img = Gray.(img)

BenchmarkTools.Trial: 
  memory estimate:  384.34 KiB
  allocs estimate:  8
  --------------
  minimum time:     1.790 ms (0.00% GC)
  median time:      2.021 ms (0.00% GC)
  mean time:        2.198 ms (1.84% GC)
  maximum time:     18.771 ms (0.00% GC)
  --------------
  samples:          2264
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 2.198
OpenCV 0.269
PIL 0.985

$ OpenCV < PIL < Images.jl$

Test 5: Applying Gaussian Blur 

img = load("img/mountainstream.png")
@benchmark gauss = imfilter(img, Kernel.gaussian(5))

BenchmarkTools.Trial: 
  memory estimate:  18.70 MiB
  allocs estimate:  603
  --------------
  minimum time:     39.857 ms (0.00% GC)
  median time:      42.566 ms (0.00% GC)
  mean time:        44.348 ms (4.84% GC)
  maximum time:     93.842 ms (0.00% GC)
  --------------
  samples:          113
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 44.348
OpenCV 4.007
PIL 33.232

$ OpenCV < PIL < Images.jl$

Test 6: Generating (Greyscale) Histogram 

img = load("img/mountainstream.png")
@benchmark edges, counts  = imhist(img,256)

BenchmarkTools.Trial: 
  memory estimate:  386.06 KiB
  allocs estimate:  5
  --------------
  minimum time:     19.395 ms (0.00% GC)
  median time:      20.976 ms (0.00% GC)
  mean time:        22.056 ms (0.10% GC)
  maximum time:     55.325 ms (0.00% GC)
  --------------
  samples:          227
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 22.056
OpenCV 1.29
PIL 1.773

$ OpenCV < PIL < Images.jl$

Test 7: Changing colorscale to HSV 

img = load("img/mountainstream.png")
@benchmark imghsv = HSV.(img)

BenchmarkTools.Trial: 
  memory estimate:  4.50 MiB
  allocs estimate:  8
  --------------
  minimum time:     6.093 ms (0.00% GC)
  median time:      6.461 ms (0.00% GC)
  mean time:        7.373 ms (7.22% GC)
  maximum time:     64.638 ms (0.00% GC)
  --------------
  samples:          677
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 7.373
OpenCV 3.675
PIL 22.607

$ OpenCV < Images.jl < PIL$

Test 8: Calculating Integral Image 

img = load("img/mountainstream.png")
@benchmark integral_img = integral_image(img)

BenchmarkTools.Trial: 
  memory estimate:  9.00 MiB
  allocs estimate:  7
  --------------
  minimum time:     4.437 ms (0.00% GC)
  median time:      5.785 ms (0.00% GC)
  mean time:        7.329 ms (16.98% GC)
  maximum time:     73.310 ms (13.81% GC)
  --------------
  samples:          681
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 7.329
OpenCV 0.226
scikit-image 37.098

Note: PIL didn't have a method to calculate integral image. Benchmarked against scikit-image instead.

$ OpenCV < Images.jl < scikit-image$

Test 9: Rotating Image (90 Degrees) 

img = load("img/mountainstream.png")
@benchmark rotated = imrotate(img, pi/2)

BenchmarkTools.Trial: 
  memory estimate:  1.13 MiB
  allocs estimate:  5
  --------------
  minimum time:     16.047 ms (0.00% GC)
  median time:      16.682 ms (0.00% GC)
  mean time:        17.877 ms (0.82% GC)
  maximum time:     59.864 ms (0.00% GC)
  --------------
  samples:          280
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 17.877
OpenCV 4.317
PIL 2.808

$ PIL < OpenCV < Images.jl$

Test 10: Corner Detection using Harris Method 

img = load("img/mountainstream.png")
@benchmark corners = imcorner(img; method = harris)

BenchmarkTools.Trial: 
  memory estimate:  70.88 MiB
  allocs estimate:  101873
  --------------
  minimum time:     75.884 ms (8.44% GC)
  median time:      83.419 ms (9.94% GC)
  mean time:        86.072 ms (10.41% GC)
  maximum time:     136.603 ms (24.07% GC)
  --------------
  samples:          59
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 86.072
OpenCV 22.802
scikit-image 112.88

$ OpenCV < Images.jl < scikit-image$

Task 11: Morphological Operation - Erode 

img = load("img/mountainstream.png")
@benchmark imge = erode(img, [5,5]) #over 5x5 1's Kernel

BenchmarkTools.Trial: 
  memory estimate:  1.13 MiB
  allocs estimate:  3
  --------------
  minimum time:     405.601 μs (0.00% GC)
  median time:      512.500 μs (0.00% GC)
  mean time:        706.785 μs (23.49% GC)
  maximum time:     71.665 ms (99.14% GC)
  --------------
  samples:          7020
  evals/sample:     1

Result:

Framework Time Taken (μs)
Images.jl 706.785
OpenCV 1190.893
scikit-image 40561.481

$ Images.jl < OpenCV < scikit-image$

Task 12: Morphological Operation - Opening 

img = load("img/mountainstream.png")
@benchmark imgc = closing(img, [5,5]) #over 5x5 1's Kernel

BenchmarkTools.Trial: 
  memory estimate:  1.13 MiB
  allocs estimate:  3
  --------------
  minimum time:     408.499 μs (0.00% GC)
  median time:      469.999 μs (0.00% GC)
  mean time:        652.171 μs (22.81% GC)
  maximum time:     33.034 ms (0.00% GC)
  --------------
  samples:          7595
  evals/sample:     1

Result:

Framework Time Taken (μs)
Images.jl 652.171
OpenCV 1587.633
scikit-image 68730.478

$ Images.jl < OpenCV < scikit-image$

Test 13: Morphological Operation - TopHat 

img = load("img/mountainstream.png")
@benchmark imgth = tophat(img, [9,9])

BenchmarkTools.Trial: 
  memory estimate:  2.25 MiB
  allocs estimate:  5
  --------------
  minimum time:     960.600 μs (0.00% GC)
  median time:      1.263 ms (0.00% GC)
  mean time:        1.687 ms (19.92% GC)
  maximum time:     28.826 ms (95.24% GC)
  --------------
  samples:          2937
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 1.687
OpenCV 1.82
scikit-image 65.544

$ Images.jl < OpenCV < scikit-image$

Test 14: Morphological Operation - BottomHat 

img = load("img/mountainstream.png")
@benchmark imgth = bothat(img, [9,9])

BenchmarkTools.Trial: 
  memory estimate:  2.25 MiB
  allocs estimate:  5
  --------------
  minimum time:     950.299 μs (0.00% GC)
  median time:      1.079 ms (0.00% GC)
  mean time:        1.482 ms (20.42% GC)
  maximum time:     23.362 ms (92.39% GC)
  --------------
  samples:          3346
  evals/sample:     1

Result:

Framework Time Taken (ms)
Images.jl 1.482
OpenCV 1.913
scikit-image 81.389

$ Images.jl < OpenCV < scikit-image$

Test 15: Segmentation- Connected Components 

img = load("img/mountainstream.png")
@benchmark markers = label_components(img)

BenchmarkTools.Trial: 
  memory estimate:  10.85 MiB
  allocs estimate:  70
  --------------
  minimum time:     9.715 ms (0.00% GC)
  median time:      10.430 ms (0.00% GC)
  mean time:        12.003 ms (9.84% GC)
  maximum time:     36.741 ms (58.12% GC)
  --------------
  samples:          416
  evals/sample:     1

End of report.

Please accept now sir.