challenges of Blackbox testing

Black Box Testing is always more complex than if you have full control over the application to test. But for all these challenges, there are alternatives available. Some of them are undemanding, some of them a little bit more complicated.

How to control the application from outside?

The very first question is, when we automate something from the outside, we first know how to control it. At first select the setup should like this:

For web applications, Selenium is the obvious choice to automate your application. For Mobile Applications , Appium is the obvious choice. It is based on the same API as Selenium, so if you know one, you would know how to use the other.

There are certain challenges in using it, but they are solvable.

Desktop Applications

Here’s another recommended API based tool  Selenium/Appium . There’s an App Driver (like ChromeDriver, EdgeDriver) for Windows applications. It helps you to automate Universal Windows Platform (UWP), Windows Forms (WinForms), Windows Presentation Foundation (WPF), and Classic Windows (Win32) apps on Windows 10 PCs.

How to Start the Application

On Windows, you have to configure and start/stop an IIS. But when do we get to know that the application is ready to use? If there is a status endpoint to query, you could use this. If not, you can query an endpoint, that has no side effect, so it doesn’t matter, that you call it multiple times.

Mobile Applications

Here Appium handles that for us so it is easy to start in mobile applications. 

Timing Issues

When you start automating an application, there are some parts, where you simply have to wait. If the APIs are providing events, when something is finished, please use them. Then you have no unnecessary wait time at all.

Resetting the Environment

It is a challenge, where the short answer is “It depends”..If you have a database, you have to restore the dump or back up your database (which probably takes ages). But again, this depends on your database.

Special Configuration

There is a chance that you need to configure your application in a specific way, that only it will get testable. But if you have to test a Micro- Service, which calls other Micro- Services, we would want to configure it in a way that it is calling some Stub/Mock-Microservices to have it also in control, what they are returning.

Keeping the State of the Applications twice

Because you can’t access the state of the application, you probably need very quickly to store also some state in your test automation. This is normal. You have to be cautious that the two states don't run into different directions. These are bugs that are hard to find, so spend some more time in designing an API around this.

It looks like this:

Shutdown of Application

So if you think your test automation ran through, all asserts were successful and it’s done? No, unfortunately not. shut down your application as gracefully as possible. So that your next scenario does not fail simply because your file got corrupted because it wasn't fully written to disk.

Conclusion

As you see, out-of-the-box testing brings some challenges with it, but if you know them, you can resolve them quite easily. It is better to have challenging test automation, than no test automation at all or doing it manually.