Autonomous QA System Walkthrough 🤖

Hi there, my name is Mark.

I'm one of the co founders of Propolis, the autonomous QA system.

I'm going to give you a quick run through of how it works.

If you're on the launch flow, you should be able to set up a swarm immediately.

So let's log in.

the swarm is an autonomous test generation engine.

It's going to explore your app systematically, it's going to surface bugs, generate tests, and as it runs again and again, it will learn of the changes in your app as you're pushing new features, continuously refreshing the test that you have so you don't have to worry about maintenance.

let's start with the weights and biases platform, which is a pilot customer of ours, as the entry point.

If you want to provide documentation, you can, but it's not necessary.

You then give it the variables and files you need so you know if you want to give it addresses or files for it to use in the platform.

and you would set up a profile.

The profile is essentially a saved chrome state, so it can use your cookies.

this is more reliable and faster than credential based.

But you're, you're welcome to do that as well.

the rest is already set up for an initial pass.

All you need to do is launch the swarm.

The run is executing right now.

You can see the live exploration as it goes.

Agent navigates the page, interacts with elements, all that stuff.

You'll get an email when it's time to log in and see the results, but I wanted to show you a completed one.

So you can see here the vast array of tests that the platform generated.

All these green ones are tests.

You can see the actions that the agents took, the bugs that they found, and how this all worked.

Once, the run is complete, they will plop the tests in a proposed section.

you're going to review the tests which will give you a video of the agent actually walking through the platform and, and the test steps that it took to get there, including the check that it's making.

So the assertion that something happened.

you can accept or reject these based on relevance or accuracy.

and then they would run on certain triggers like commits or deploys, and those would show up in your test runs page.

this is a way for you to see the different actions the agents took and make sure that everything's going well.

while the swarm is running, it's also going to find bugs.

so a byproduct of the swarm is, you know, did I see a bug while I was going after a certain objective and servicing it.

Up to you there as well.

the alternative approach here is for you to build your own tests.

We have a few ways to do this.

You can prescribe an objective to the agent and help it walk through that.

You can upload a video of you actually taking steps and it will make a test based on that.

or you can and build it step by step yourself.

so the core value prop here is that the swarm generates tasks autonomously, services bugs during exploration.

these tools give you sort of a manual way to do that if you're not getting the tests you desperately need.

and the goal is just production ready tests without manual test authoring and a continuous loop of maintaining and auto generating new ones as your app evolves.

Happy to answer any questions.