Life in the clouds, Googie style (1962).

Using AWS

by: burt rosenberg
at: university of miami
date: january 2021

Overview

From the 202 edition of this course (Jan-May 2020) the course has gone fully to programming on AWS EC2 instances. The code will of course be on our subversion servers, as has been the practice since the 152 edition (Jan-May 2015). You access the EC2 instance from any computer using ssh, scp and sftp, using public-key style authentication.

Many platforms are possible for accessing your instances. I will assume a laptop or desktop type computer running Linux, Windows or MacOS. I have been a Mac user since they switched to OSX, since the body of that operating system is unix, but had a nicer interface than the the free unix's.

I also have extensive experience with Windows NT, having been a kernel maintainer for the NT Kernel while I worked at Citrix, but I have not kept up with it. I run a large number of Ubuntu machines for the department. I also have experience with other varieties of Linux, but no so much Linux on the desktop. It is my familiarity with Ubuntu that is why I ask that your EC2 instances be Ubuntu.

Amazon Web Services

The idea of cloud computing is to provide easily scalable computing, on demand, on a pay-as-you-go basis. The first CSC424 to use AWS was the 122 edition (2012). You will create a computing instance from a compute instance, (EC2), and storage instance, (EBS), with the internetworking glue of key-pairs for logging in and Security Groups for controlling network access.

Log into aws.amazon.com, you could use your Amazon (shopping) credentials. They also have IAM, an authentication management system that will preserve your full credentials for a root account on AWS, and you can create roles that are less privileged. You might also set up 2FA using something like DUO. for instance, can create EC's but cannot mess with your billing information. I have not explored this yet, and run in the root account (not the best).

Creating an EC2 Instance

From the EC2 Dashboard select Instance Launch. The Launch Status page appears.

Installing and svn co

Navigate to the EC2 Instances page and when the Instance Status is running, checkbox select it and select Connect. You will get a connection string similar to, 
    ssh -i "aws-csc424.pem" ubuntu@100.24.236.162
Cut and paste this into a terminal after cd'ing to the keys directory where the file aws-csc424.pem has been placed. 
raritan:keys ojo$ pwd
/Users/ojo/svn/classes/csc424-202.svn/class/keys
raritan:keys ojo$ chmod go-r aws-csc424.pem 
raritan:keys ojo$ ls -lt
total 16
-rw-------@ 1 ojo  staff  1692 Apr 19 15:50 aws-csc424.pem
-r--------@ 1 ojo  staff  1692 Apr 19 11:56 csc424-keypair.pem
raritan:keys ojo$ ssh -i "aws-csc424.pem" ubuntu@100.24.236.162
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-1057-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Sun Apr 19 20:26:41 UTC 2020

  System load:  0.08              Processes:           86
  Usage of /:   13.6% of 7.69GB   Users logged in:     0
  Memory usage: 14%               IP address for eth0: 172.30.1.238
  Swap usage:   0%

0 packages can be updated.
0 updates are security updates.



The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@ip-172-30-1-238:~$ sudo apt-get update
Hit:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic InRelease
Get:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]

...

Get:27 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [7176 B]
Get:28 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2764 B]
Fetched 18.6 MB in 4s (4861 kB/s)

ubuntu@ip-172-30-1-238:~$ sudo apt-get install subversion build-essential
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu cpp cpp-7 dpkg-dev fakeroot g++ g++-7 gcc gcc-7 gcc-7-base gcc-8-base
  libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libapr1 libaprutil1 libasan4 libatomic1 libbinutils
  libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-7-dev libgcc1 libgomp1
  libisl19 libitm1 liblsan0 libmpc3 libmpx2 libquadmath0 libserf-1-1 libstdc++-7-dev libstdc++6 libsvn1 libtsan0 libubsan0
  linux-libc-dev make manpages-dev
Suggested packages:
  binutils-doc cpp-doc gcc-7-locales debian-keyring g++-multilib g++-7-multilib gcc-7-doc libstdc++6-7-dbg gcc-multilib autoconf
  automake libtool flex bison gdb gcc-doc gcc-7-multilib libgcc1-dbg libgomp1-dbg libitm1-dbg libatomic1-dbg libasan4-dbg
  liblsan0-dbg libtsan0-dbg libubsan0-dbg libcilkrts5-dbg libmpx2-dbg libquadmath0-dbg glibc-doc bzr libstdc++-7-doc make-doc
  db5.3-util libapache2-mod-svn subversion-tools
The following NEW packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu build-essential cpp cpp-7 dpkg-dev fakeroot g++ g++-7 gcc gcc-7 gcc-7-base
  libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libapr1 libaprutil1 libasan4 libatomic1 libbinutils
  libc-dev-bin libc6-dev libcc1-0 libcilkrts5 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-7-dev libgomp1 libisl19
  libitm1 liblsan0 libmpc3 libmpx2 libquadmath0 libserf-1-1 libstdc++-7-dev libsvn1 libtsan0 libubsan0 linux-libc-dev make
  manpages-dev subversion
The following packages will be upgraded:
  gcc-8-base libgcc1 libstdc++6
3 upgraded, 45 newly installed, 0 to remove and 73 not upgraded.
Need to get 45.7 MB of archives.
After this operation, 175 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y

Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic-updates/main amd64 gcc-8-base amd64 8.4.0-1ubuntu1~18.04 [18.7 kB]
Get:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libstdc++6 amd64 8.4.0-1ubuntu1~18.04 [400 kB]
Get:3 http://us-east-1.ec2.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libgcc1 amd64 1:8.4.0-1ubuntu1~18.04 [40.6 kB]

...

Setting up build-essential (12.4ubuntu1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
ubuntu@ip-172-30-1-238:~$  svn co --username burt svn://svn.cs.miami.edu/classes/csc424.202 css424.svn
Authentication realm: <svn://svn.cs.miami.edu:3690> department
Password for 'burt': ************

-----------------------------------------------------------------------
ATTENTION!  Your password for authentication realm:

   <svn://svn.cs.miami.edu:3690> department

can only be stored to disk unencrypted!  You are advised to configure
your system so that Subversion can store passwords encrypted, if
possible.  See the documentation for details.

You can avoid future appearances of this warning by setting the value
of the 'store-plaintext-passwords' option to either 'yes' or 'no' in
'/home/ubuntu/.subversion/servers'.
-----------------------------------------------------------------------
Store password unencrypted (yes/no)? yes
A    css424.svn/class
A    css424.svn/class/examples
A    css424.svn/class/examples/hello-world
A    css424.svn/class/examples/hello-world/Makefile

...

A    css424.svn/burt/proj4/mypasswords.txt
A    css424.svn/burt/proj4/utils.c
Checked out revision 303.
ubuntu@ip-172-30-1-238:~$

Stopping and terminating the EC2 instance

You do not need to keep your EC2 instance running all the time, accruing rental fees. You can either stop the instance and pay a lower EBS rental fee on the disk or terminate the instance and then you released all resource and will owe nothing more.

If you terminate, you can do the creation, install, and svn so again on a fresh instance. It is not too bad, and it will you will incur the lowest possible charges over the course. I think it might be possible that if you are a new AWS user, the first year free tier will make it free. Else it is under $150 for the semester.

Every time you stop or terminate, you looks the IP address that was assigned to the instance. I generally deal with this by having macros in my makefiles, and they need to be updated to the IP address currently assigned. You can avoid loosing the IP by using Elastic IP. It will hold the IP address for you even when you are not running yoru EC2 instance. But they do charge rent on an IP address that is being held in this way.

Let's talk $$$'s

From EC2 pricing, and EBS pricing on April 2020.

The Free Tier is for new users their first 12 months. Free is,

All others,

Remote editing

Rather then edit in vi on the ec2 instance, programmer editors can edit by sftp, or scp can be used. In this method. You put a copy of your repo both on your local machine and on the ec2 instance. You edit the local copy and either sftp or scp the file to the ec2 instances.

The ssh command uses the .ssh/config to configure shortcuts. Place in your conf file write something like, 

		Host aws-csc424-s
		Hostname 18.208.120.70
		User ubuntu
		IdentityFile ~/csc424.svn/burt/keys/aws-csc424.pem
The Host can be whatever name you want. The hostname is the IP address of the instance, and you will have to update that often. The username ubuntu is the default username is ubuntu. The IdentityFile gives the full path to the pem file to be used for authentication.

Then you can log into you machine with: 

    ssh aws-csc424-s
You can copy the local file hello.c to your machine wit: 
   scp hello.c aws-csc424-s:

A good programmers editor for Mac that includes SFTP is BBEdit. Using this, there is a save-to, open-from sftp option. The BBEdit software uses the .ssh/config options, so the name of the server will be (in this case) aws-csc424-s.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

author: burton rosenberg
created: 18 apr 2020
update: 18 jan 2024