Gitlab combines code storage with, issue tracking, testing and deployment tools. It's designed for the complete development workflow from start to finish. We are going to play with the Gitlab pipelines and setup a complete automated workflow based on branches.
1) Setup Gitlab. Either sign-in to the free hosted version, or download and install your own version.
2) Create a new Project and download the code repository to your local machine.
3) Create a backend codebase. In my example i've decided to create a simple Python/Flask application which renders a single page:
backend/requirements.txt
Flask==0.12
backend/__init__.py
import os, sys lib_path = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'lib') sys.path.insert(0, lib_path) from .server import app if __name__ == "__main__": app.run()
backend/server.py
import os
from flask import Flask, render_template, send_from_directory
static_dir = os.path.join(os.path.dirname(__file__), 'static')
template_dir = os.path.join(os.path.dirname(__file__), 'templates')
app = Flask(__name__, static_folder=static_dir, template_folder=template_dir)
@app.route('/')
def index():
return render_template('index.html')
@app.route('/api')
def test():
return 'Hello people!'
if __name__ == '__main__':
app.run(host='0.0.0.0', port=3000, debug=True)
<p>it works</p>
You can run the application using the commands:
pip install -r requirements.txt python server.py
4) In the root of the project we can now add support for Google AppEngine using:
app.yml
runtime: python27
threadsafe: true
handlers:
- url: /
script: backend.app
- url: /static
static_dir: backend/static
skip_files:
- ^(.*/)?#.*#$
- ^(.*/)?.*~$
- ^(.*/)?.*\.py[co]$
- ^(.*/)?.*/RCS/.*$
- ^(.*/)?\..*$
- ^(.*/)?.*\_test.(html|js|py)$
- ^(.*/)?.*\.DS_Store$
- ^.*bower_components(/.*)?
- ^.*node_modules(/.*)?
- ^.*jspm_packages(/.*)?
5) Make sure you have installed the Google Cloud SDK. We can now deploy this manually to AppEngine using the commands:
gcloud init gcloud app deploy
But we want to automate this process, how?
6) Create a .gitlab-ci.yml file in the root of your project containing the steps to build the backend, and deploy the code:
.gitlab-ci.yml
build_backend:
image: python
stage: build
script:
- pip install -t backend/lib -r backend/requirements.txt
- export PYTHONPATH=$PWD/backend/lib:$PYTHONPATH
artifacts:
paths:
- backend/
deploy:
image: google/cloud-sdk
stage: deploy
environment:
name: $CI_BUILD_REF_NAME
url: https://$CI_BUILD_REF_SLUG-dot-$GAE_PROJECT.appspot.com
script:
- echo $GAE_KEY > /tmp/gae_key.json
- gcloud config set project $GAE_PROJECT
- gcloud auth activate-service-account --key-file /tmp/gae_key.json
- gcloud --quiet app deploy --version $CI_BUILD_REF_SLUG --no-promote
after_script:
- rm /tmp/gae_key.json
This will use the current branch name as the version, so if you commit code to master, your appengine url will be https://master-dot-projectname.appspot.com
7) Last step before committing your changes to Gitlab (and running the pipeline), you need to set the variables. Go to:
Gitlab Project > Settings > CI/CD Pipelines > Secret Variables
And fill out the following variables:
* GAE_KEY: Secret key for a user with the required permissions.
* GAE_PROJECT: Name of the target AppEngine application.
You will need to get these from Google Cloud at:
Google Cloud Project > IAM & Admin > Service Accounts > Create Service Acccount
Google Cloud Project > IAM & Admin > Service Accounts > Create Service Acccount
8) If you push the code to Gitlab, you should now see the steps running one-by-one:
Click on a failed job to see the output of the errors:
9) Last step is to add some unit/functional tests. I've decided to go use PyUnit and Selenium tools to make the process easier. First we update our project code:
backend/requirements.txt
Flask==0.12 chromedriver-installer==0.0.6 selenium==3.4.2
import unittest
from backend import app
class Test(unittest.TestCase):
def test(self):
result = app.test_client().get('/api')
self.assertEqual(result.data.decode('utf-8'), 'Hello people!')
qa/browser.py
import os
import unittest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
DRIVER = os.getenv('DRIVER', 'headless_chrome')
BASE_URL = os.getenv('BASE_URL', 'http://backend:3000')
SELENIUM = os.getenv('SELENIUM', 'http://localhost:4444/wd/hub')
def get_chrome_driver():
desired_capabilities = webdriver.DesiredCapabilities.CHROME
desired_capabilities['loggingPrefs'] = {'browser': 'ALL'}
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument(
"--user-data-dir=/tmp/browserdata/chrome \
--disable-plugins --disable-instant-extended-api")
desired_capabilities.update(chrome_options.to_capabilities())
browser = webdriver.Chrome(
executable_path='chromedriver',
desired_capabilities=desired_capabilities)
# Desktop size
browser.set_window_position(0, 0)
browser.set_window_size(1366, 768)
return browser
def get_headless_chrome():
desired_capabilities = webdriver.DesiredCapabilities.CHROME
desired_capabilities['loggingPrefs'] = {'browser': 'ALL'}
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument(
"--user-data-dir=/tmp/browserdata/chrome \
--disable-plugins --disable-instant-extended-api \
--headless")
desired_capabilities.update(chrome_options.to_capabilities())
browser = webdriver.Remote(
command_executor=SELENIUM,
desired_capabilities=desired_capabilities)
# Desktop size
browser.set_window_position(0, 0)
browser.set_window_size(1366, 768)
return browser
DRIVERS = {
'chrome': get_chrome_driver,
'headless_chrome': get_headless_chrome
}
def get_browser_driver():
return DRIVERS.get(DRIVER)()
qa/test_example.py
import unittest
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from browser import get_browser_driver, BASE_URL
class ExampleTestClass(unittest.TestCase):
def setUp(self):
print("BASE_URL", BASE_URL)
self.driver = get_browser_driver()
def tearDown(self):
self.driver.quit()
def test_page_title(self):
self.driver.get(BASE_URL)
print("driver.title", self.driver.title)
self.assertIn("Gitlab AppEngine CI", self.driver.title)
elem = self.driver.find_element(By.NAME, "search")
elem.send_keys("Selenium")
# elem.send_keys(Keys.RETURN)
# assert "No results found." not in driver.page_source
def test_javascript_text(self):
self.driver.get(BASE_URL)
wait = WebDriverWait(self.driver, 10)
wait.until(EC.visibility_of_element_located(
(By.CSS_SELECTOR, 'div#output')))
elem = self.driver.find_element(By.ID, 'output')
self.assertIn("JavaScript + gulp too!", elem.text)
if __name__ == "__main__":
unittest.main()
python -m unittest discover -s backend python -m unittest discover -s qa
10) Let's automate the unit/functional tests by adding the following lines to the .gitlab-ci.yml file:
.gitlab-ci.yml
test_unit:
image: python
stage: test
script:
- export PYTHONPATH=$PWD/backend/lib:$PYTHONPATH
- python -m unittest discover -s backend
test_functional:
image: python
stage: test
services:
- selenium/standalone-chrome
script:
- export PYTHONPATH=$PWD/backend/lib:$PYTHONPATH
- SELENIUM="http://selenium__standalone-chrome:4444/wd/hub" BASE_URL="https://$CI_BUILD_REF_SLUG-dot-$GAE_PROJECT.appspot.com" DRIVER="headless_chrome" python -m unittest discover -s qa
Hope that helps you get set up!
View the full working code here:
https://gitlab.com/kim3/gitlab-appengine-ci
No comments:
Post a Comment