Moodle_Plugin_Development

Visit Our YouTube Channel

Behat Acceptance Tests in Moodle Plugin Development.

Requirements to Features.

As a Moodle plugin developer for over seven years now, I have not really been concerned with automated testing of plugin code as it appeared to be only a requirement for Moodle core developers. In fact, providing automated tests is not a requirement for plugins submitted to the directory – see the Plugin contribution checklist page at https://docs.moodle.org/dev/Plugin_contribution_checklist. For most of my previous developments, my clients were happy with me providing written test scripts as they usually would not have had large testing teams if any dedicated testing resource at all. Many of my clients may have not really wanted the additional development time and/or cost of developing the tests – despite any argument that it could save time and cost at a later stage particularly the bug fixing stage.

However, I recently was contracted to develop a plugin with the additional requirement that I provide both PHPUnit tests and Acceptance tests using the Behat framework as per the Moodle core. Considering that SimpleTest support was completely removed from Moodle 2.4, released in December 2012, to be replaced by PHPUnit – introduced in version 2.3 - and Behat testing appearing in Moodle 2.5 in May 2013, I find myself behind the times quite considerably. Things may have been a little better if I had over these years been exposed to PHP testing in other developments but I have not.

I will not cover the Moodle PHPUnit testing any further in this article apart from the fact that setting up the testing environment for both test types proved a bit of a challenge. I will also not cover the environment setup, you are best to attempt to follow the instructions on the Moodle site, https://docs.moodle.org/dev/PHPUnit for PHPUnit and https://docs.moodle.org/dev/Running_acceptance_test for Behat. To avoid issues, ensure you get the latest version of the ChromeDriver for your OS and your installed Chrome browser.

Be warned that if you are like me and decide to Behat test your installation by running only one feature file, you may get the dreaded “ The base URL (http://behatmoodle.local) is not a behat test site, ensure you started the built-in web server in the correct directory or your web server is correctly started and set up” message. I suggest you run the test against auth/tests/behat/login.feature which appears to work to determine if your installation works.

Acceptance Testing using the Behat framework is currently used for regression testing in core Moodle. You would notice this if you spend anytime examining the feature test files for many of the core plugins. I, on the other hand, am developing the plugin and then producing tests, a little bit different. Also I am definitely not using Test driven testing (TDD) methodology which would have required the tests be developed first.

I am however writing this article as I attempt the development of the Behat tests for my development. So in this article, I will attempt to get from the plugin requirements to a list of the features that would be used for the basis for the testing. The plugin discussed in this article is not my client’s project (confidentiality) but one that I am developing for my currently in development book.

The client’s high level requirement is :

“ Students must be able to record online video submissions for assignments as an individual or a member of a group.”

This means that I needed to develop an a ssign ment submission plugin ( https://docs.moodle.org/dev/Assign_submission_plugins) for the client. If I were to write a test now, it would fail.

Test No

Test Description

User

Steps

Pass

1

Record and submit a video for an assignment.

Student

  1. Login

  2. Navigate to Course

  3. Click on ‘New Assignment’

  4. Click on the ‘Add submission’

  5. Click ‘Record Your Submission’ button

  6. Record your response;

  7. Click ‘Save Submission’

  8. Click ‘Submit assignment’ button (if available)

  9. The submission status should now read ‘Submitted for grading’ .

N

The first thing we can note is that there are some pre-requisites to the test – the background – in Behat (Gherkin) parlance. Let’s think about it :

  1. Student must have a user account;

  2. A course needs to exist;

  3. The Student must be enrolled on the course;

  4. The assignment must exist;

  5. The student must have permission to make a submission;

  6. The assignment needs to allow video submissions.

Pre- requirements 1 to 5 are standard Moodle functionality and it is likely that the test steps are already defined. Moodle provides a tool to view the defined step definitions available – just navigate to ‘ Site administration → Development → Acceptance testing’ (admin/tool/behat/index.php). I personally don’t it but it is worth using when you get to actually writing the tests.

Pre-requirements 5 and 6 are interesting in that if the assignment module tests for them, it will not be testing it specifically for our plugin so we may have to test that too. Additionally, we would need to test the functionality with the student as an individual and as a member of a group so the test needs to involve two assignments.

For the relevant assignment to exist, we need to allow the course administrator – teacher or manager - to create and manage an assignment that will be available to the student. This would need to happen prior to the student test. So here goes :

Test No

Test Description

User

Steps

Pass

1

Add an assignment that requires an ‘Online Video’ submission.

Teacher

  1. Login

  2. Navigate to Course

  3. Click ‘Turn Editing on’

  4. Select a section and click the ‘Add an activity or resource’ link and choose ‘Assignment’ and Click the ‘Add’ button

  5. Enter the ‘Assignment name’ and ‘Description’ in the relevant input fields

  6. In the ‘Submission types’ section of the form, select the ‘Online Video’ submission type

  7. Click the ‘Save and return to course’ button

  8. Your new assignment should appear in the relevant section.

N

2

Record and submit a video for an assignment.

Student

N

Well if you were to follow the steps, you would OK up to the point – step 6 – and then the test would fail. The ‘Online Video’ submission type would not exist.

Again we have some pre-requisites to run this test:

  1. Teacher must have a user account;

  2. A course needs to exist;

  3. The teacher must be enrolled as a editing teacher on the course;

  4. The assignment needs to allow video submissions.

All four pre-requirements are standard core Moodle functionality so will have the relevant steps available for the Teacher’s background. However to test submission as an individual and as a group member, two assignments would have to be created for the test.

Out test failed because the ‘Online Video’ submission type option was not available. So how do we make it available? We need the site admin to ‘activate’ the plugin, so now:

Test No

Test Description

User

Steps

Pass

1

Activate the ‘Online Video assignment’ submission plugin.

Admin

  1. Login

  2. Navigate to ‘ Site administration -> Plugins -> Activity modules -> Assignment -> Submission plugins -> Manage assignment submission plugins’ (mod/assign/adminmanageplugins.php?subtype=assignsubmission) page

  3. Click the closed eye icon in the ‘Hide/Show’ column for the ‘Online Video’ plugin;

  4. The eye icon should now be an open eye.

N

2

Add an assignment that requires an online recorded video submission.

Teacher

N

3

Record and submit a video for an assignment.

Student

N

The test again fail due to the missing pre-requirement which is that the plugin will need to have been installed. We can safely assume that the admin user account will exist. We could add a new step to our tests to include the installation and configuration (if necessary) but we will ignore that for our purposes and assume the plugin is installed and configured for our tests.

I don’t know about you but I like to start my plugin development with a set of skeleton files that will allow me to install a plugin before it does any real work. It is likely that even then, the test will fail until some required functionality is provided.

Now if I was writing a functional manual acceptance test as usual, the test plan would be:

  1. Admin activates the plugin;

  2. Teacher creates an activity to be completed by the student as an individual;

  3. Teacher creates an activity to be completed by the student as a group member;

  4. Student completes the activity as an individual;

  5. Student completes the activity as a group member.

We could extend the tests a little to prove that the teacher can view and grade the submissions which might be a good idea to ensure that the submission views work as expected.

  1. Teacher grades the individual submission;

  2. Teacher grades the group submission.

We could also test that a student who is not a member of the group cannot make a submission and we could have various permutations of submissions options to test but we can assume that functionality that relates to all submissions have been covered off by core assignment tests. Indeed it is possible that the client may stipulate further requirements that could add further testing being needed.

We could now write a feature test that would cover all the steps. If we combine all the common pre-requisites for our tests we end up with:

  1. Teacher must have a user account;

  2. Student must have a user account;

  3. The course needs to exist;

  4. The teacher must be enrolled as a editing teacher on the course;

  5. The student must be enrolled on the course;

All the other prerequisites require action by another user, either the admin or the teacher, and so have to form part of the tests.

Writing one test to cover off all the steps is possible but doing that would be messy and long increasing the chances that any slight change to the code will make the test fail. It is obvious that we have two ‘streams’ of tests, the individual and the group submissions. Both streams require the same pre-requisites, background in Behat/Gherkin parlance. There is an additional pre-requisites if we are going to test group functionality and that is :

  1. A course group must be defined.

  2. The student must be assigned to a group.

Our two streams can now become 2 test scenarios. However, arguably we could have a situation where we have more than 2 scenarios e.g. for the individual submissions we could have ‘Individual with No Click Submission’ and ‘Individual with Click Submission’ to test. Thinking ahead then, we notice that the requirement to enable the plugin would be common to all the scenarios. Instead of it forming part of the tests, we can make it a pre-requisite for all subsequent tests. This would require that a testing step be defined like the core tests we alluded to previously. Does Moodle come with a predefined test to enable an assignment submission plugin or any plugin for that matter? A search reveals no such luck. That means we have to define a step to define as per the ‘Writing new acceptance test step definitions’ ( https://docs.moodle.org/dev/Writing_new_acceptance_test_step_definitions) page. Doing that is outside the scope of this article but lets assume that the step is defined.

If we step back to the beginning and with a little rewording, we can use the client's requirement as the feature to be tested. Now our test plan looks something like this :

Feature

Students are able to record online video submissions for assignments as an individual or a member of a group.

Background

  • Plugin must be enabled;

  • Teacher must have a user account;

  • Student must have a user account;

  • The course needs to exist;

  • The teacher must be enrolled as a editing teacher on the course;

  • The student must be enrolled on the course;

  • A course group must be defined.

  • The student must be assigned to a group.

Individual Scenario

Group Scenario

  • Teacher creates an Online Video assignment to be completed by the student as an individual;

  • Student completes the Online Video assignment as an individual;

  • Teacher grades the individual submission;

  • Teacher creates an Online Video assignment to be completed by the student as a group member;

  • Student completes the Online Video assignment as a group member.

  • Teacher grades the group submission.

This is simplistic. We could for example split the 2 scenarios into two features and in cases where there is quite a bit to test, it may make sense to do so.

We are now in the position to actually attempt to code a feature file for our tests and we will tackle that in our next article.

Finally, you may have recognised that my manual functional test steps resemble user stories and that is how I previously determined workflows to quote for work when a client approached me with some high level requirement. So in another article I cover off going from user stories to acceptance tests.

Back Home