Meatchland

Ultimate SSH and Git setup

December 11, 2016 @ 4:42 am

Chapter 1: Introduction

This series examines what I have found to be the Ultimate SSH and Git setup. It assumes you have familiarity with the following:

  • Web Servers – how to get around, what the public directory is
  • A familiarity with terminal commands and bash

I am embracing Git as my primary workflow

I have finally committed to using Git to manage my websites. I have been flying commando with Transmit (FTP) and Atom’s FTP plugin remote-sync. While this is quick, and requires very little know-how – it does place my sites at risk for no versioning and fallback. As well as being able to really test things locally. Essentially my workflow has been as follows:

  1. Develop initial website on a live testing server – typically a subdomain  (e.g. dev.domain.com)
  2. Once the client is happy – we set it live (these days just by changing the folder to be the new public folder – lickity split)
  3. All future changes are managed on the server (truth be told – if its a major revision, I will port the site to a test environment again. e.g. dev.domain.com). Once changes approved, port them over to the live.

Many problems with this – and the most glaring is – my development is exposed to the public when I work live. Not good for the client – errors can take a site offline if something breaks.

I have used Git in the past

I have used Git in the past – ’cause I wanted to understand it. Everyone’s talks about it as the bees knees – and if you aren’t doing it there is something wrong with you. : ) As a teacher too – I really wanted to learn Git to be up to snuff and give my students all of what is possible.

My whole experience was WHY DOES IT HAVE TO BE SO FRICK’N DIFFICULT!! The pros never seemed to outweigh the costs of having to set all of this up. And the documentation is definitely challenging for the newbie – unless you have a true programming background – its confusing how it does its magic (where to connect what, etc). It also seemed like overkill for me – since I am essentially a one-man band. I could see how large teams could benefit from being able to check out files, merge and correct conflicts and such – but for someone who manages all of it – it felt like using a bazooka when all I needed was a knife.

Give Git a Chance

Granted, I get complaints like this from my students about HTML, CSS and JS – so I get it. It’s something new I am learning, and I need to give it the proper chance to percolate in the brain. And this weekend I finally did. The future seems bright, and I am excited to finally be tackling and using this beast on my sites. Time will tell how it all pans out.

As a teacher I love simplifying things – breaking it down so others can absorb it. I am also self taught – and when I read articles online I am always thinking about – how can I explain this to others such that they really get it. So I have challenged myself in creating this series in my blog – to help others–AND because I know a few months from now when I have to do all of this shit again – I will be like-what goes where again? As you will see – a lot of this is like following a recipe. You don’t have to memorize the steps – just refer to them when you need them. And in no time you will have a pie. Easy as pie?

What is Git

Git is a version control system  – a fancy way of saying you can:

  • roll things back if you make a mistake.
  • create branches without messing with the master version of your website
  • work in teams and each team member can work on their branch without worrying about messing things up. When a teammate merges their branch, Git will let you know if there are differences in the source – and allow you to correct any problems on the fly. Everyone then pulls these changes so all instances are in sync

Pretty awesome for all of the above.

What is Github

Github is a place to store you code if you choose to (it’s free). It uses Git, and gives you the ability to manage your web content. You could even use Github as your web server – so I have heard. But most people use it to develop their code, and then push the master version to their live hosted website.

Github != Git

When I first heard of Github, I thought that was the only way to do it. I collapsed the two – thinking they were one and the same. But you can install Git on your web server (webhost) and push directly to it–cutting out the middle man (Github). This is what makes sense to me as a one-man-band. If you are working on a team – you may want to consider using Github (or alternatives like bitbucket) to work on your projects, and then use Web Hooks to push your changes to the live server. I am gonna explore this one further – just to say I can. For now, I am gonna convert all of my existing websites and call it a day.

Exposure and Security Concerns

My other concern was security. I always saw Github as exposed to the public – and I was afraid of exposing my usernames and passwords. Git actually takes care of this in the .gitignore – you can say which files to ignore – and these never get posted to Github. Github is exposed to the public – which promotes open source. This means someone could find your code – clone it and work off of your system. if you do have a private project you can use Bitbucket (or alternative). I think  read somewhere that Github also offers this feature – but at cost. Don’t quote me on that one. Point is, there are options to making it private.


Chapter 2: Terminal, SSH and Keyless Password

This series examines what I have found to be the Ultimate SSH and Git setup. It assumes you have familiarity with the following:

  • Web Servers – how to get around, what the public directory is
  • A familiarity with terminal commands and bash

The Terminal, Bash, and Shell Scripts

In order to leverage Git, you essentially are moving over to the terminal (e.g. shell, command line) – you feel like a regular hacker in this world. And like my feelings with Git as a whole – felt like overkill when I have this wonderful GUI that can do all of it and more. Yet there is something that is pretty slick about using the terminal. Especially once you get the hang of it. You feel like you are one with the machine. And processes on the server are so much faster than over FTP. Because you are actually working on that machine. Using its resources. Very powerful. Very magical. Very cool. Bash is a text file script you can create that will execute a series of terminal commands. So if there is something you are doing a lot of – bash can be used to automate these tasks. If you have not gotten at least the basics of terminal yet – you should definitely dig in to the basics – otherwise the rest of this will be challenging to understand. It gives you the “Why Water is Wet.”

SSH

SSH (Secure SHell) – it allows you to log into your Webserver (or any computer) and use terminal on that computer. This is the most important piece – as Git is run through the terminal. You can, set up a GUI for Git. But I would suggest using the GUI apps once you understand fully how git is working.

Most shared hosting services will let you turn on a single SSH account for all of your content. If you have a virtual or dedicated server – its pretty standard – and sometimes on by default. Regardless, contacting your hosting support will let you know how to enable it, and get you connected.

Once you have SSH enabled, and you have your credentials – you will open a terminal and type something that looks like: ssh username@domain.com, press return and provide your password once prompted.

  • ssh is the prefix that tells the terminal we are Secure SHelling into a remote computer.
  • username is the username on your account (if you only have one shared account – it is probably root)
  • domain.com is your domain. Sometimes the domain won’t cut it – so it’s your web server IP address.

Enable SSH in the WHM Cpanel

If you don’t know what WHM Cpanel is–you are most likely on a shared server (standard) and can ignore this part. If you have WHM Cpanel manager (Virtual and Dedicated Servers), you will probably have to manually enable SSH.

  1. Log into WHM
  2. Choose Manage Shell Accounts
  3. Check Normal Shell radio button next to the account
    1. It auto saves for you – nothing else to do here.
  4. Open terminal and test SSH access
    1. ssh username@ip-or-domain.com
    2. provide account password

Keyless SSH Password

Many of the processes we run with Git will go through SSH. If you have to enter your password every time you make a change – you will lose your mind. So instead, we use Keyless Passwords which consist in 2 parts

  1. You generate a local encrypted pair of keys (private and public)
  2. The public key is placed on your web server and acts as a way to verify you are you.

Public Key: This gets stored on the remote host proving you have access to the account. Original also stays on your local computer.

Private Key: this remains on your system. The public key is used to match the private encrypted key – identifying you as valid access to the server(s)

Part A) Create encrypted, local public and private keys

You only have to do this once. You can use the same encrypted key pairs for all connection – it identifies you individually as valid. So you can skip to Part B.

When I say execute, I mean type the command and press return.

  1. open terminal 
  2. Head to your home folder (should be there by default), to be sure execute:
    cd ~/
  3. List the contents (including hidden files -a and list details -l)
    ls -al
  4. If a .ssh folder already exists – you may have already created keys – so be careful that you do not overwrite someone elses keys. You may want to rename the .ssh folder just in case and start from scratch. (mv .ssh/ .ssh-bakup)
    If there is no .ssh folder, or you are now creating from scratch make a new .ssh folder.
    mkdir .ssh
  5. Move into the .ssh folder
    cd .ssh
  6. Create the Keys: where your_email@example.com is your personal email – this is optional – but I believe helps on teams in identifying who is who when logging into the server.
    ssh-keygen -t rsa -b 4096 -C “your_email@example.com”
  7. When it asks for a filename, don’t type anything in, just press return
  8. Give it a secure SSH passphrase (Highly recommended)
    There is a concern if your system is comprimised, someone can get to the public and private keys without using the password and now they have SSH to your server.If you do decide to add a passphrase, make sure your OS saves it as a keychain – otherwise it will ask you every time you SSH. It should automatically save to your OSX keychain – use the following instructions if it does not.
    https://help.github.com/articles/working-with-ssh-key-passphrases/
  9. Add your new key to the SSH-Agent – the program that keeps track of all of this stuff.
    ssh-add ~/.ssh/id_rsa

Part B) Add public key to your server

  1. Open terminal and SSH into your remote host
    ssh username@ip-or-domain.com and password (until you finish this you have to give your pass)
  2. Make sure the remote server has a .ssh/ in the account root directory.
    ls -al
  3. if it does not exist create one
    mkdir .ssh
  4. Open another terminal (local/home)
    cd ~/
  5. Run the following in your local terminal, where user@ip-or-domain.com is your personal SSH
    ssh username@ip-or-domain.com ‘cat >> ~/.ssh/authorized_keys’ < ~/.ssh/id_rsa.pub
    The first time you run the following command the remote server will ask for your password.
    example: ssh enspyred@104.168.148.183 ‘cat >> ~/.ssh/authorized_keys’ < ~/.ssh/id_rsa.pub
  6. Permissions are important, otherwise it won’t work. So back on the remote server terminal window
    chmod 700 ~/.ssh
    chmod 600 ~/.ssh/authorized_keys

Upon success you should be able to log out, and log back in without terminal asking for a password.


Chapter 3: Installing Git

This series examines what I have found to be the Ultimate SSH and Git setup. It assumes you have familiarity with the following:

  • Web Servers – how to get around, what the public directory is
  • A familiarity with terminal commands and bash

You will want to install Git locally, and make sure your web server has Git installed and enabled.

Install Git on your web server

Make sure your web server has Git installed and enabled. Most do these days, and you can either do it manually, or just ask your web host support to help you install it. You can run a test to see if it is already there:

  1. Open Terminal
  2. SSH into your remote host
  3. Enter the following:
    git –version
  4. If you get a git version returned you are good to go (e.g. git version 1.7.3.4)

To be fair, from what I have read–not many shared hosting companies have Git installed. And you may need to jump through some hoops to install it. I would contact your web host support – they can tell you right away what is possible.

Install Git Locally

Easiest to follow their instructions for your system

https://www.git-scm.com/book/en/v2/Getting-Started-Installing-Git


Chapter 4: Set Up Git Directly On Your Web Server

This series examines what I have found to be the Ultimate SSH and Git setup. It assumes you have familiarity with the following:

  • Web Servers – how to get around, what the public directory is
  • A familiarity with terminal commands and bash

I have to give credit to this website – ’cause he was the first person to actually break this down into terms that made sense to me – and it just worked. It is possible that I have been playing with this long enough to begin to grok the Klingon language – so there is that too. I changed the order to optimize the flow – moving from live to local that is.

You can Git any folder on your web server. If you are building a web application, this often includes the root directory (one directory up from your public_html  or htdocs folder), but its not required.

1) Download a copy of your website

If you do not have a local copy of your website, now’s the time to download it. This includes all dependencies (images, css, js).I used my FTP program to do this – it allows me to see hidden files, most FTP programs do (Transmit) – and to sync missing files. If you are familiar with rsync, and your server supports it – you can use command line to sync your server.

Truth be told, on my server I am serving up some videos, and I didn’t want Git to manage these. If you look at most web servers there are a bunch of files that don’t matter to you, but help your server run. You will use a .gitignore file to tell Git to leave those out of the mix. In other words, you can download only the parts you plan on using Git to manage. Keep in mind – your folder structure needs to be identical to the server. So don’t move your CSS folder to a new location, download them as is. And, don’t forget hidden files and folders that are needed (files and folders that have a dot in front of their name e.g. .htaccess).

2) Setup a bare repo on your web server (remote server)

  1. SSH into your remote host
  2. Create a new folder in our rot directory (right above your public directory), where domain.com is your domain.com.
    mkdir domain.com.git && cd domain.com.git
    This will make the dir, and CD you into the directory
    note: when I first read this I thought it was odd we did not place this into the folder we are using Git on – you will see that we alter this to point to whichever directory we are running git on. This is just so we don’t mess up the existing website.
  3. Create a Bare Git Repository (repository without an initial workspace)
    git init –bare
  4. cd into the hooks directory
    cd hooks
  5. Create a new bash post-receive hook that will be triggered whenever an update is pushed to this new bare repository.
    touch post-receive
  6. Open post-receive to edit in the terminal
    nano post-receive
  7. Paste the following code: (where “/path/to/your/htdocs/directory” is the actual path to the directory you are using git)
    #!/bin/bash
    GIT_WORK_TREE=/path/to/your/htdocs/directory git checkout -f
  8. Exit and Save
    control + x
    y
  9. Make sure its executable by updating its permissions
    chmod +x post-receive

3) Initialize Git in Your Local Copy (local)

  1. Make a noticeable change to your local website and save (I prefixed the title with the current time – great for testing)
  2. Using Terminal – cd into the local copy of your website
    tip: you can figure out the directory by dragging the folder to the terminal. So type cd,  and then drag the folder to the terminal – it will add the path where the terminal cursor was. A quick way to get to the GUI folder – especially if its on an external Volume deep in the woods.
  3. Initialize Git
    git init
  4. Add the contents of the current directory to the git repository (the period is intentional)
    git add .
  5. Commit all your newly added files to the repo
    git commit -m “My First Commit”

4) Create a .gitignore file (local)

The .gitignore text file tells git which files and folders to leave out of the synching process. Create this file in the main folder of your local git repository. If you have files and folders in your local copy that you do not wish to push to the server – then you should add them to this file. They are line separated definitions that look like the following:

#this is a comment and is ignored, great for organizing your ignore files and folders
.env
sensitive-passwords.txt
img/videos
*.psd
# the star is a wildcard – so *.psd says any file that ends in .psd

Note: once you push your changes to your server – it will add the git ignore to your server as well. So you should add the server side directories you wish to ignore as well.

See my example .gitignore from Hostwinds server – I also wanted to keep out my large video files. Not that you have to – I just don’t plan on managing those through Git as my future videos will either be on YouTube, Vimeo or Join.me.

5) Push local to live (local)

  1. Add the remote hook to your local git
  2. git remote add live ssh://user@domain.com/path/to/domain.com.git
    1. where live will be the name we will use to push to that server. You can have multiple servers if you like – each with their own name.
    2. and  user@domain.com is your SSH user and domain or IP
    3. and domain.com.git is the git folder you made on your web server
    4. tip: if you are not sure what the path is to your remote git folder, SSH in to your remote host. CD to the folder, and use the following command:
      pwd
      This will print the current working directory for you. Copy and paste
  3. Push the changes we made earlier to the server
    git push -u live master

Chapter 5: The Ultimate SSH, Git, Gulp Workflow

What is Gulp?

It automates processes…

RSync Live site changes to local

As a site grows on its own you are probably gonna want to download the current version of it. Into you existing local setup.

Example
rsync -hvrPt –stats username@ip-or-domain.com:/path/to/remote/dir/ ./

The above assumes you have passwordless ssh – otherwise it asks for pass?

Flags
-chavzP = forces overwrite so it mirrors live.
-hvrPt = If you wish to not overwrite local files that are newer than the server.

SSH
root@104.168.148.183

Remote Dir
/path/to/remote/dir/

Local Dir
– could probably just use the period .
– have not tested it though
./

 

Local Gulp

watch
Sass Conversion and Min – no vanilla CSS
JavaScript Concat and Minification

Local Testing Server

Optional Local Database
Keep this separate – we can always download the db and run it locally if need be.

Running Local Server
gulp serv

Gulp Git Push When Ready to go live

gulp gitsend -m “Message for change” [Git add, commit, push]

LIGHTBOX CONTENT