Flutter Automated Testing. Step-by-Step Guide

Testing is one of the most important phases of mobile app development. You can’t build a high-quality app without testing it. The testing process requires precise planning and execution, but it’s also the most time-consuming part of development. The Flutter framework provides comprehensive support for Flutter automated testing of mobile apps.
Manually testing mobile apps can be hard, especially if your app has many features. Automated tests help to ensure that your app performs correctly before you publish it while maintaining your feature and bugfix velocity. Let’s find out more about Flutter app development and Flutter app testing.

Categories of automated testing

Unit tests

A unit test evaluates a single function, method, or class. The goal of Flutter unit testing is to verify the correctness of a unit of logic under a variety of conditions. External dependencies of the unit under test are generally mocked. Unit tests usually don’t read from or write to the disk, render to the screen, or receive user actions from outside the process running the test.

Here’s how to write a unit test:

Step 1: Create a test file

For this example, let’s start by creating two files: validator.dart and validator_test.dart.

The validator.dart file will contain the class you want to test and will reside in the lib folder. The validator_test.dart file will contain the tests themselves and live in the test folder.

When you’re finished, the folder structure should look like this:

my_app/
  lib/
    validator.dart
  test/
    validator_test.dart

Now we have to curate our class to test it. Next, we need a “unit” to test.

Remember: A “unit” is another name for a function, method, or class.

For this example, let’s create a Validator class that will be responsible for simple password and email validation as well as for enums, which we’ll use to recognize the validation status.

enum PasswordValidationResults {
  VALID,
  TOO_SHORT,
  EMPTY_PASSWORD,
}

enum EmailValidationResults {
  VALID,
  NON_VALID,
  EMPTY_EMAIL,
}

class Validator {
  final emailRegExp = RegExp(r"^[a-zA-Z0-9.a-zA-Z0-9.!#$%&'*+-/=?^_`{|}~]+@[a-zA-Z0-9]+.[a-zA-Z]+");

  PasswordValidationResults validatePassword(String password) {
    if (password.isEmpty) {
      return PasswordValidationResults.EMPTY_PASSWORD;
    }
    if (password.length < 6) {
      return PasswordValidationResults.TOO_SHORT;
    }
    return PasswordValidationResults.VALID;
  }

  EmailValidationResults validateEmail(String email) {
    if (email.isEmpty) {
      return EmailValidationResults.EMPTY_EMAIL;
    }
    if (!emailRegExp.hasMatch(email)) {
      return EmailValidationResults.NON_VALID;
    }
    return EmailValidationResults.VALID;
  }
}

Step 2: Write a test for the class

Now let’s find out how to write a test for the class.
For starters, inside the validator_test.dart file, write the first unit test. Tests are defined using the top-level test function, and you can check if the results are correct using the top-level expect function. Both of these functions come from the test package.

import 'package:flutter_test/flutter_test.dart';
import 'package:demo/validator.dart';

void main() {
  test('Validator Test', () {
    final validator = Validator();

    expect(validator.validatePassword(''), ValidResults.EMPTY_PASSWORD);
    expect(validator.validatePassword('passw'), ValidResults.TOO_SHORT);
    expect(validator.validatePassword('validPass'), ValidResults.VALID);
  });
}

Also, we can assemble tests in a group.

If you have several related tests, you can combine them using the group function provided by the test package.

void main() {
  group('Validator', () {
    final validator = Validator();

    test('Password Test', () {

      expect(validator.validatePassword(''), PasswordValidationResults.EMPTY_PASSWORD);
      expect(validator.validatePassword('passw'), PasswordValidationResults.TOO_SHORT);
      expect(validator.validatePassword('validPass'), PasswordValidationResults.VALID);
    });

    test('Email Test', () {
      expect(validator.validateEmail(''), EmailValidationResults.EMPTY_EMAIL);
      expect(validator.validateEmail('email.com'), EmailValidationResults.NON_VALID);
      expect(validator.validateEmail('email@hmail.1'), EmailValidationResults.NON_VALID);
      expect(validator.validateEmail('email@hmail.com'), EmailValidationResults.VALID);
    });
  });
}

Step 3: Run the tests

Now you can run your tests and make sure you wrote the code that you want.
The Flutter plugins for Android Studio and Visual Studio Code support tests. Using these plugins is often the best option when writing tests because they provide the fastest feedback as well as the ability to set breakpoints.

To do X:

Android Studio

1. Open the counter_test.dart file.
2. Select the Run menu.
3. Click the Run ‘tests in counter_test.dart’ option.
4. Alternatively, use the appropriate keyboard shortcut for your platform.

Visual Studio Code

1. Open the counter_test.dart file.
2. Select the Debug menu.
3. Click on Start Debugging.
4. Alternatively, use the appropriate keyboard shortcut for your platform.

Widget tests

A widget test (in other UI frameworks referred to as a component test) tests a single widget. The goal of a widget test is to verify that a widget’s UI looks and interacts as expected. Testing a widget involves multiple classes and requires a test environment that provides the appropriate widget lifecycle context.

For example, the widget being tested should be able to receive and respond to user actions and events, perform layout, and instantiate child widgets. A widget test is, therefore, more comprehensive than a unit test. However, as with a unit test, the environment for a widget test is much simpler than the full-blown UI.

Step 1: Create a UI

Let’s create a UI for later flutter UI testing. We’ll create a simple login form that will contain a custom widget (MyTextWidget), two text input fields, and one send button, which will be responsible for validating X (which we created previously) and sending data to an API.

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: LoginWidget(title: 'Flutter Demo Home Page'),
    );
  }
}

class LoginWidget extends StatefulWidget {
  LoginWidget({Key key, this.title}) : super(key: key);

  final String title;

  @override
  _LoginWidgetState createState() => _LoginWidgetState();
}

class _LoginWidgetState extends State {
  var _emailTextController = TextEditingController();
  var _passwordTextController = TextEditingController();

  var _passErrorTxt;
  var _emailErrorTxt;

  Validator validator = Validator();

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        backgroundColor: Colors.deepPurple,
        title: Text('Login'),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.spaceBetween,
          children: [
            Padding(
              padding: EdgeInsets.only(top: 50, left: 50, right: 50),
              child: Column(
                children: [
                  MyTextWidget('My label'),
                  TextField(
                    key: Key('emailTextField'),
                    controller: _emailTextController,
                    decoration: InputDecoration(
                      errorText: _emailErrorTxt != null ? _emailErrorTxt : null,
                    ),
                  ),
                  TextField(
                    key: Key('passTextField'),
                    controller: _passwordTextController,
                    decoration: InputDecoration(
                      errorText: _passErrorTxt != null ? _passErrorTxt : null,
                    ),
                  ),
                ],
              ),
            ),
            Padding(
              padding: EdgeInsets.symmetric(vertical: 20, horizontal: 60),
              child: RaisedButton(
                key: Key('buttonKey'),
                onPressed: () => _login(),
                color: Colors.amber,
                child: Container(
                  height: 45,
                  width: double.infinity,
                  child: Center(
                    child: Text(
                        'SEND'
                    ),
                  ),
                ),
              ),
            )
          ],
        ),
      ),// This trailing comma makes auto-formatting nicer for build methods.
    );
  }

  @override
  void dispose() {
    //we should dispose our controllers to avoid some leaks
    _emailTextController.dispose();
    _passwordTextController.dispose();
    super.dispose();
  }

  _login() {
    setState(() {
      _emailErrorTxt = null;
      _passErrorTxt = null;
    });

    EmailValidationResults emailValidationResults = validator.validateEmail(_emailTextController.text);
    PasswordValidationResults passwordValidationResults = validator.validatePassword(_passwordTextController.text);
    if (emailValidationResults == EmailValidationResults.VALID && passwordValidationResults == PasswordValidationResults.VALID) {
      //login successful
    }

    setState(() {
      switch (emailValidationResults) {
        case EmailValidationResults.NON_VALID:
          _emailErrorTxt = 'Invalid email';
          break;
        case EmailValidationResults.EMPTY_EMAIL:
          _emailErrorTxt = 'Empty email';
          break;
        case EmailValidationResults.VALID:
          _emailErrorTxt = null;
      }
      switch (passwordValidationResults) {
        case PasswordValidationResults.TOO_SHORT:
          _passErrorTxt = 'Password is too short';
          break;
        case PasswordValidationResults.EMPTY_PASSWORD:
          _passErrorTxt = 'Empty password';
          break;
        case PasswordValidationResults.VALID:
          _passErrorTxt = null;
      }
    });
  }
}

class MyTextWidget extends StatelessWidget {
  final String labelText;

  MyTextWidget(this.labelText);

  @override
  Widget build(BuildContext context) {
    return Text(
      labelText,
      textAlign: TextAlign.center,
      style: TextStyle(
        color: Colors.deepOrange,
        fontSize: 16,
      ),
    );
  }
}

Step 2: Create a testWidgets test

After creating a login widget, we have to create a testWidgets test.

With a widget to test, begin by writing your first test. Use the testWidgets function provided by the flutter_test package to define a test. The testWidgets function allows you to define a widget test and creates a WidgetTester to work with.

This test verifies that MyTextWidget displays a given label text.

import 'package:flutter_test/flutter_test.dart';

void main() {
  testWidgets('MyTextWidget has a label', (WidgetTester tester) async {
    
  });
}

Step 3: Build the widget using WidgetTester

Next, build MyTextWidget inside the test environment using the pumpWidget method provided by WidgetTester. This method builds and renders the provided widget.
Then create a MyTextWidget instance that displays “MyLabel” as the label.

After the initial call to pumpWidget, WidgetTester provides additional ways to rebuild the same widget. This is useful if you’re working with a StatefulWidget or animations.

For example, tapping a button calls setState, but Flutter won’t automatically rebuild your widget in the test environment. Use one of the following methods to ask Flutter to rebuild the widget.

• tester.pump()
  Triggers a rebuild of the widget after a given duration.
• tester.pumpAndSettle()
  Repeatedly calls pump for the given duration until there are no longer any frames scheduled. It essentially waits for all animations to complete.

These methods provide fine-grained control over the build lifecycle, which is particularly useful while testing.

Step 4: Search for your widget using a finder

With a widget in the test environment, search through the widget tree for the label Text widget using a finder. This allows you to verify that your widget is being displayed correctly.

For this purpose, use the top-level find method provided by the flutter_test package to create the finders. Since you know you’re looking for text widgets, use the find.text method.

void main() {
  testWidgets('MyTextWidget has a label', (WidgetTester tester) async {
    await tester.pumpWidget(MyTextWidget('MyLabel'));

    // Create the Finders.
    final labelFinder = find.text('MyLabel');
  });
}

Step 5: Verify the widget using a matcher

Finally, verify the title and messages. Text widgets appear on the screen with the help of the matcher constants provided by flutter_test. Matcher classes are a core part of the test package. They provide a common way to verify if a given value meets expectations.
Ensure that the widgets appear on the screen exactly once. For this purpose, use the findsOneWidget Matcher.

void main() {
  testWidgets('MyTextWidget has a label', (WidgetTester tester) async {
    await tester.pumpWidget(MyTextWidget('MyLabel'));

    // Create the Finders.
    final labelFinder = find.text('MyLabel');

    // Use the `findsOneWidget` matcher provided by flutter_test to verify // that the Text widgets appear exactly once in the widget tree.
    expect(labelFinder, findsOneWidget);
  });
}

Now let’s verify that our LoginWidget contains two input fields and one button:

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

import 'package:demo/main.dart';

void main() {
  testWidgets('LoginWidget contains two input fields and one button', (WidgetTester tester) async {
    await tester.pumpWidget(MaterialApp(home: LoginWidget(),));

    final inputFieldsCount = find.byType(TextField);

    expect(inputFieldsCount, findsNWidgets(2));
  });
}

Note that I pumped the Material app, where I provided an argument called LoginWidget instead of pumping LoginWidget directly. That’s because LoginWidget is a StatefulWidget and depends on MediaQuery.

To provide MediaQuery for testing widgets, wrap your StatefulWidget to the MaterialApp widget.

For more information about flutter widget testing, see the Flutter documentation.

Integration tests

An integration test tests a complete app or a large part of an app. The goal of a Flutter integration test is to verify that all widgets and services being tested work together as expected. Furthermore, you can use integration tests to verify your app’s performance.
Generally, an integration test runs on a real device or an OS emulator, such as iOS Simulator or Android Emulator. The app under test is typically isolated from the test driver code to avoid skewing the results.

Unit tests and widget tests are handy for testing individual classes, functions, and widgets. However, they generally don’t test how individual pieces work together as a whole or capture the performance of an application running on a real device. These tasks are performed with integration tests.
Integration tests are carried out in two phases: first, they deploy an instrumented application to a real device or emulator; then they “drive” the application from a separate test suite, making sure everything is correct along the way.

To do X:

Step 1: Add the flutter_driver dependency

Use the flutter_driver package to write integration tests. Add the flutter_driver dependency to the dev_dependencies section of the apps’s pubspec.yaml file.
Also, add the test dependency in order to use actual test functions and assertions.

dev_dependencies:
  flutter_driver:
    sdk: flutter
  test: any

We’ve already created an app to test as well as the test files.

However, unlike unit and Flutter widget tests, integration tests do not run in the same process as the app being tested. Therefore, we need to create two files that reside in the same directory. By convention, the directory is named test_driver.

1. The first file contains an “instrumented” version of the app. The instrumentation allows you to “drive” the app and record performance profiles from a test suite. This file can have any name that makes sense. For this example, we created a file called test/app.dart.
2. The second file contains the test suite, which drives the app and verifies it works as expected. The test suite also records performance profiles. The name of the test file must correspond to the name of the file that contains the instrumented app, with _test appended. Therefore, create a second file called test/app_test.dart.

Now let’s instrument the app. This involves two steps:

1. Enable the Flutter driver extensions.
2. Run the app.

Add this code inside the test_driver/app.dart file:

import 'package:flutter_driver/driver_extension.dart';
import 'package:demo/main.dart' as app;

void main() {
  // This line enables the extension.
  enableFlutterDriverExtension();

  // Call the `main()` function of the app, or call `runApp` with any 
// widget you are interested in testing.
  app.main();
}

Step 2: Write the tests

Now that you have an instrumented app, you can write tests for it. This involves four steps:

1. Create Serializable Finders to locate specific widgets.
2. Connect to the app using the setUpAll function before the tests run.
3. Test important scenarios.
4. Disconnect from the app using the teardownAll function after the tests are complete.

// Imports the Flutter Driver API.
import 'package:flutter_driver/flutter_driver.dart';
import 'package:test/test.dart';

void main() {
  group('MyApp', () {
    // First, define the Finders and use them to locate widgets from the // test suite. Note: the Strings provided to the `byValueKey` method must // be the same as the Strings we used for the Keys in step 1.

    final emailTextFieldFinder = find.byValueKey('emailTextField');

    FlutterDriver driver;

    // Connect to the Flutter driver before running any tests.
    setUpAll(() async {
      driver = await FlutterDriver.connect();
    });

    // Close the connection to the driver after the tests have completed.
    tearDownAll(() async {
      if (driver != null) {
        driver.close();
      }
    });

    test('Field investigation', () async {
      // Use the `driver.tap` method to find the input field.
      await driver.tap(emailTextFieldFinder);
      // verify that your input field is empty.
      await driver.waitFor(find.text(''));
      // Use the 'driver.enterText' method to enter the text to your input field.
      await driver.enterText('email@email.com');
      // verify that your input field contains entered text from the step above.
      await driver.waitFor(find.text('email@email.com'));
    });
  });
}

Step 3: Run the tests

Now that you have an instrumented app and a test suite, you can run the tests. To do so, you’ll need either an emulator (for Android) or a simulator (for iOS), or you’ll need to connect your computer to a real iOS or Android device. Make sure you’ve added the Flutter command to your path variable (just run Flutter doctor in your terminal). If you get a message that the command can’t be found, follow the steps described in the link below. Just choose your operating system and add Flutter to your path:

Now that you’ve added the Flutter command to your path variable, you can run tests. Go to your project folder, then run the following command from the root of the project:

flutter drive --target=test_driver/app.dart

This command:

1. Builds the –target app and installs it on the emulator or device.
2. Launches the app.
3. Runs the app_test.dart test suite located in test_driver/ folder.

BLOC Test Coverage

Now I’m going to create the BLOC architecture for our app. It will contain LoginBloc (which will be responsible for business logic), LoginState (which will be responsible for the bloc’s state management) and LoginEvent (which will be responsible for sending events to the Bloc component).

Let’s code it.

The first thing you need to do is add the flutter_bloc: ^2.0.1 dependency to pub spec.yaml (a Flutter package that helps implement the BLOC pattern). Also, you have to extend Equatable in order to check if two instances of AuthState are equal. Now I’m going to add Bloc to my app:

import 'package:equatable/equatable.dart';
import 'package:meta/meta.dart';

@immutable
abstract class AuthState extends Equatable {

  @override
  List

Our first test is just testing that our initialState is what we expect… nothing too fancy. The second test is just a sanity check as well to make sure that when close is called, the state of the bloc is not updated.

After running the Flutter test, our first two tests should pass!

Now we’ll modify our test file to make sure the bloc works correctly.

import 'package:flutter_test/flutter_test.dart';

import 'package:mockito/mockito.dart';
import 'package:my_memories/bloc/auth_bloc.dart';
import 'package:my_memories/bloc/auth_event.dart';

import 'package:my_memories/bloc/auth_state.dart';
import 'package:my_memories/repository/auth_repo.dart';

class MockUserRepository extends Mock implements AuthRepo {}

void main() {
  AuthBloc authenticationBloc;
  MockUserRepository userRepository;

  setUp(() {
    userRepository = MockUserRepository();
    authenticationBloc = AuthBloc(userRepository);
  });

  tearDown(() {
    authenticationBloc?.close();
  });

  test('initial state is correct', () {
    expect(authenticationBloc.initialState, InitialState());
  });

  test('close does not emit new states', () {
    expectLater(
      authenticationBloc,
      emitsInOrder([InitialState(), emitsDone]),
    );
    authenticationBloc.close();
  });

  group('LoggedIn', () {
    test(
        'emits [initial, loading, login] when bool value is persisted',
            () {
          final expectedResponse = [
            InitialState(),
            LoadingState(),
            LoginState(true),
          ];

          when(userRepository.login(any, any)).thenAnswer((_) => Future.value(true));
          
          expectLater(
            authenticationBloc,
            emitsInOrder(expectedResponse),
          );

          authenticationBloc.add(LoginEvent('email@mail.com', 'AdminAdmin'));
        });
  });
}

These tests are very simple: we set up the expectations and then add the event. In this case, we expect that AuthBloc will yield InitialState first of all, and then after calling LoginEvent with an email and password, we’ll receive LoadingState and LoginState after calling the face repository.

For more detailed information, check out the official Flutter documentation

Final thoughts

This article is a starting point for exploring Flutter testing. It briefly introduces you to how Flutter mobile app testing works. We’ve described ways of testing a mobile application using Flutter. We can implement tests both to verify that the code meets technical requirements.
There are trade-offs between different kinds of testing. Remember that a well-tested app has many unit and widget tests tracked by code coverage, plus enough integration tests to cover all important use cases.