GPU_for_cuTile

Hey guys, how's it going?

I just kind of wanted to give a high level of what this PR is doing.

this creates this folder infra AWS GPU and it really just stands up a GPU using aws.

So for people like myself or on a Mac or various other machines in the world don't have this built in gpu, how do we actually like get this code to run?

Right, the qtar.

So this is what I did was I created this folder and there's quite a few files in here, but ignored the majority of them.

I created a readme and an agent md.

So this essentially is the agents MD that I used with Claude and Codex to come up with the bash scripts and the provisioning of the EC2 instance to be as seamless as possible.

So I could just start running Q tile almost immediately.

So I wrote this out pretty easily, right?

First you got to have the repo and then you got to install these couple packages, the AWS cli.

For people that already have aws, this is going to be a no brainer.

and then after you install those guys, it kind of gives you a couple different workflows.

I come from the perspective that nobody, they don't even have an AWS account if that's true.

They just go to root user security credentials and create an access key.

Now the first thought like I had was I don't want root access and a security key hanging around.

That's true.

So what ends up happening is you provision this bootstrap admin.

So you'll have this access key open for a minute.

But once you create the bootstrap IM sh.

So once you run this, so you run these sequentially, you'll upload this, you'll run aws, configure this bootstrap admin, it will ask you to paste in what you just created inside the profile default region, blah blah blah.

In 3C right here is it will run the bootstrap and then it will create a QTILE profile which has least privilege access.

At the very end it will prompt you to revoke the bootstrap key.

So the key you legitimately just created inside of AWS will now be revoked.

It could be up for a total of 30 seconds, potentially that you have this root access.

The reason is you need some kind of root access to be able to create iam, roles.

So, so you have to have that first.

Now after you've ran this command, you will just have a QTILE profile which has least access.

So like can't go into billing, credit cards, the whole nine yards.

So that's like my first gotcha is it will delete the root admin that you created and only use this profile.

Moving forward after that you'll just copy this TFRS example and essentially that just stands up a couple of the main variables that you'll need.

So US East 1, the Qtop profile that you just created, the GX5, that's a dollar per hour, it's one of the cheapest ones I could find.

Project name and then the SSH user.

after you run that, it will create the profile for you and then you can check your GPU vpc, VCPU quota.

Mine was zero at first.

Some people might already have.

Essentially you have to be allowed to have the GPU, right?

So you can check your quota.

If it's zero, you need to request update.

Like eight is generally it, you need greater than four to run the G5.

You run this service quota.

If you don't have it, you can obviously go inside of the AWS GUI and request it manually.

For me, it took like a day, to get provisioned.

Sometimes it's a couple minutes, not a big deal.

And then here's the nice part is it's three bash scripts which has been broken down to update.

SH will do your entire provisioning of your terraform instances.

So it'll run variables tf.

Everything that you just created with the qtol profile will get stood up.

It will create the IAM policy, it will stand up essentially everything.

so then you'll have an EC2 inside of, you'll have your GPU instance inside of AWS and then you can run the smoke test.

So I run this run qtile smoke.

Essentially what this does is it puts in all the requirements from your guys's repo and it shoves it into the single bash script.

So that way we can just run the code example that's on the main readme and you can see that the GPU is actually working.

And then at the very end the down sh, very simple.

We just don't want to get build anymore.

You run down SH and it will kill everything that just got provisioned inside of AWS.

You don't have to manually to delete the EC2, the IAM profiles, the policies, nothing.

It will go in well, the IAM policy, the QTile will hang around, but it doesn't charge you any money.

And then at the very end you can test it on a branch, fork a test branch, wherever the case is.

So, like, I'm just putting in my repo, right?

So this is like whatever your repo is, whatever branch you're running this from, probably main.

And then you're just running the smoke test.

At the end you will have ran the example qtil sh and it will be connected to your ec2.

and then you can run down and kill the whole thing.

I hope this explains it, guys.

Appreciate it.

Bye.