I attended Serverless Days in Sydney Today. Overall it had a good sense of community, the venue and food was top notch. This is a community run conference and there’s always good representation from the main serverless cloud providers here. *Cough* Google/AWS/Microsoft *Cough*. Alibaba made an appearance too.
Ben Kehoe – a Cloud Robotics Research Scientist at @iRobot gave us lessons learned in applying this mindset to more than just web applications. He goes over how as engineers we need to shift our mindset to focus on delivering business value and not get caught up on feeling like the centre of the business. Any code your write is a liability. That’s why I advocate for lean test code that adds value.
On a side note, anyone who gets to work in robotics professionally is super cool in my books. My favourite presentation is one where I present Tappy McTapface – a robot for mobile app testing. You can watch a previous talk by Ben on a similar idea here:
Did you miss out on the action?
Some of the speakers have given their presentations at previous conferences/meetup groups. Like Jessica Flanagan at YOW! Data this year. And Denis Bauer at YOW! Perth last year.
I Previously live streamed the Serverless Sydney meetup group and the Node user group on twitch (however these video’s are only stored on twitch for up to 60 days post live stream).
Serverless Icon Showdown
Who has the best logo? There’s Google Cloud Functions, AWS Lambda Functions, Microsoft Azure Functions and Alibaba Cloud Function Compute. Handsdown I think Microsoft Azure functions is the best logo here.
Are you using Serverless in production yet? what are some of the challenges you are facing? How should testers get up to speed with this technology?
Have you heard of micro services? This is like that but even smaller. What’s the smallest service you could possible deploy? a single function. Server less still uses a server, but the infrastructure is abstracted away by your cloud provider of choice. If no one is hitting your API, there’s no server running and costing you money. Cloud providers spin up an instance when needed as soon as something calls your function.
If you are interested in learning more, you should go to Severless Sydney on Saturday the 27th of August. Tickets are $69. https://sydney.serverlessdays.io
How do you integrate a server less function with a mySQL backend?
const mysql = require('mysql'); const connectionName = process.env.INSTANCE_CONNECTION_NAME || 'boglogger:australia-southeast1:boglogger2'; const dbUser = process.env.SQL_USER || 'DEMOUSER'; const dbPassword = process.env.SQL_PASSWORD || 'DEMOPASSWORD'; const dbName = process.env.SQL_NAME || 'bog_logger'; const mysqlConfig = { connectionLimit: 1, user: dbUser, password: dbPassword, database: dbName, }; if (process.env.NODE_ENV === 'production') { mysqlConfig.socketPath = `/cloudsql/${connectionName}`; } // Connection pools reuse connections between invocations, // and handle dropped or expired connections automatically. let mysqlPool; exports.logBog = (req, res) => { // Initialize the pool lazily, in case SQL access isn't needed for this // GCF instance. Doing so minimizes the number of active SQL connections, // which helps keep your GCF instances under SQL connection limits. if (!mysqlPool) { mysqlPool = mysql.createPool(mysqlConfig); } mysqlPool.query('SELECT NOW() AS now', (err, results) => { if (err) { console.error(err); res.status(500).send(err); } else { res.send(JSON.stringify(results)); } }); mysqlPool.query('INSERT INTO bogs_logged(time, device_uuid, type, amount, floater) VALUES(NOW(), "e5c15033-42b3-487d-b6c8-17a1b592e9bd", 2, "S", TRUE)', (err, results) => { if (err) { console.error(err); res.status(500).send(err); } else { res.send(JSON.stringify(results)); } }); };
If you want to run your own API, you will need to create a google cloud SQL instance and add a user. You can use these commands to create a table and user.
--create tables and insert some test data CREATE TABLE bogs_logged(time DATETIME NOT NULL, device_uuid VARCHAR(36) NOT NULL, type INT(1));
--create addBogs user CREATE USER 'addBogs'@'%' IDENTIFIED BY '********'; GRANT SELECT, INSERT ON bog_logger.bogs_logged TO 'addBogs'@'%';
The humble spreadsheet is one of the most versatile digital tools that any tester has in their toolkit. I’m working on a data analytics project at InsightTimer where we are doing event tracking across our android and iOS apps using MParticle and BigQuery. I want to do some data analyst on the events coming through for new android builds. “Basic Sanity” testing of events is my goal. My first step towards this is to create a Google Sheet dashboard that integrates with BiqQuery via a javascript-esk language called App Scripts. App scripts is an amazing extension of the humble spreadsheet, it really adds a lot of extra functionality. I’m going to walk you through how you can get started with a similar project of your own.
1. Sign up for a google cloud trial
I used a test account; sammy.test.cloud@gmail.com to explore this for you and I collected screenshots along the way.
There’s a whole bunch of public data sets for you to explore
We are going to explore the hacker_news datasets in this example. This is actually a collection of the data (comments and articles) from this Hacker News website.
Previewing the table is a handy feature for getting an idea of the structure of the data
The preview gives you some sample data, an idea of columns and a way to explore
If you wanted to find the top ranked comments for author ‘4ad’ you can run a query like the following (remove the \ characters if you are copy pasting this query):
SELECT
author,
id,
ranking,
time_ts,
text,
parent
FROM \`bigquery-public-data.hacker_news.comments\` WHERE author = '4ad' AND deleted IS NULL ORDER BY ranking desc, time_ts LIMIT 10;
2. Setting up Google Sheets with App Script
Create a new sheet in your drive folder, I’ve named mine BigQueryTest. Open up the tools > Script Editor:
Save, the first time you save it will ask you to name your project
I ended up calling mine BigQueryAppScriptIntegration, original I know
Run the onOpen function. The first time you run this app script it will ask you to set up permissions
Review the permissions
select your account
More Permissions
Whew, done. Now if you navigate back to your spreadsheet you should see some added menu items:
What happens when you click on the Prepare sheet… function?
Huh, fancy that. Our spreadsheet is updated
Then how about the Generate step-by-step function?
That’s pretty cool a new tab is added
You don’t need to understand google’s code to explore this. I find it’s better to learn by starting from scratch. This code gives you a guideline for what you can achieve but you will have a better understanding of how app script works by building out your own.
3. Enable App Script APIs in your google cloud project
Replace the XXXXX string with your project id, you get this from your google cloud console
Replace the query with
'SELECT COUNT (DISTINCT id) FROM [bigquery-public-data:hacker_news.comments]'
Save and run the runQuery function. You will also need to set up permissions again the first time you run this.
Huh, that’s a new screen. I clicked on “Advanced” to see a similar permission screen I had previously encountered
Dam. I got another error message:
When I click on the details it gave me a handy url to navigate to
I went to the suggested URL and it turned out I need to enable another API somewhere else. That makes me thing I did it in the wrong spot in step 3, maybe that step wasn’t needed?
I wait a few moments and go back to my app script. I click on “Run Query”. No error messages this time. If you go to your drive folder you should see a new sheet called “Big Query Results” with one row of data:
You can test out this result is roughly correct by running the query directly in your google cloud query editor:
Now you are all set up with your API integrations. Next lesson will be building out your own dashboard that puts this set up into practice.