The difficulty of software testing: many modes, languages, platforms

The difficulty of software testing: many modes, languages, platforms

When developing our free jacquard drawing program ArahPaint, we sometimes get frustrated by apparent lack of progress.

ArahPaint4

If you open ArahPaint4, 5 or 6, there is not much difference: mostly the same tools and menus.

We do this on purpose, so old customers can continue to use the program in the same way as they are used to.

ArahPaint5

But there are a lot of changes under the hood. The other source of frustration are the bugs, which still occur in spite of all the testing. So we will give you an example of our check list for even such simple tools like drawing a line, circle or a rectangle.






ArahPaint6

The functionality only available in ArahPaint6 will be marked with (6), so you will see that there is some progress:

  • Does it work on all color modes: colormap 8-bit, true color 24-bit, RGBA 32-bit (6)?
  • For 24-bit and and 32-bit images, does it also work in antialiased mode (6)?
  • Does it work in main image and in selection (layer)?
  • Is the operation properly symmetrical (circle equal on all quadrants, line the same if drawn from the point A to B or from the point B to A)?
  • Does it work in repeat mode?
  • For pen size 1, does it work with "thick lines" option?
  • Does auto-scrolling work during drawing?
  • Does it allow change of zoom level during drawing?
  • Does it allow change of drawing position during drawing?
  • Does it offer sensible pen preview while moving the mouse before drawing?
  • Is the draw preview correct during the drawing (same as final draw operation)?
  • Is the draw view correct in all view modes (zoom in/out, repeat on/off, grid on/off, integer view on/off, real density = non square image pixels)?
  • Does it work with different x and y pen sizes? (6)
  • Does it work in "keep aspect" mode, where pen x and y dimensions are calculated automatically based on image/fabric density? (6)
  • Does it work with parametric / numeric parameters instead of mouse clicks?
  • Does it respect protected colors (not draw over locked colors)?
  • Does it work with all pen shapes (circle, square, diamond, left/right diagonal, horizontal/vertical line) (6)
  • Does it work on images larger than 32.000 pixels? (6)
  • Can it draw with brush image instead of draw color?
  • Does it work in repeat layer mode (where selection is extended over several repeats, but we can still draw in correct repeat)? (6)
  • Does it work with gradients (we have 12 gradient modes)? (6)
  • Do gradients work with foreground or background color as transparent color? (6)
  • Do gradients work with all gradient fill types (random dots, shading lines, continuous two tone shading) (6)
  • Does it work well with graphical tablets (pen behaves differently from the mouse)
  • Does undo work?
  • Does redo work?
  • Is it fast enough for all realistic use cases?
  • Does it consume a lot of memory for the operation?
  • Does it free the memory after it is done, so program does not slow down after prolonged use?
  • Is function properly explained in the manual?
  • Is it properly translated in 15 languages?
  • Does the keyboard shortcut work?

Then we have a lot of global testing for the whole program, which might also be affected by single drawing operations:

  • Does automatic saving work? (6)
  • Does restore from automatic saved image work, in case of crash? (6)
  • Does it work well in different resolutions (1024x786, 1920x1080, 2840x2160, 5120x2880)
  • Does it work well with different dpi (on small laptops with high resolution screen)? (6)
  • Does it work in dark mode (are icons and text readable)? (6)
  • Can it use national characters of supported languages in the filenames?
  • Does it work on all platforms (Linux, Windows and Mac)? (6)
  • Does it work on different operating systems of each platform (Linux: SuSE, Ubuntu, Mint...; Windows XP, Vista, 10, 11; MacOS Mojave, Catalina, Big Sur, Monterey)?
  • Does it work on all CPU architectures (Intel 32 bits and 64 bits, Mac Intel and ARM / Apple Silicon M1, M2...)?

As you can see, most of these test conditions can appear in any combination of view, draw option or platform choice. So despite all the testing, users sometimes find untested situations, and we are doing our best to fix them.

要查看或添加评论,请登录

社区洞察

其他会员也浏览了