The API I'm testing sometimes respond with different error codes and questions, how can I used 'Examples' and match assertion to check one OR other? [duplicate] - integration-testing

Files for the scenario
All the files are on same directory.
title-update-request.json
{id: 12, name: 'Old Hello', config:[{username: 'qwe', password: 'tyu'},{username: 'abc', password: 'xyz'}]}
title-update-response.json
{id: 12, name: 'New Hello', config:[{username: 'qwe', password: 'tyu'},{username: 'abc', password: 'xyz'}]}
title-update-error-request.json
{id: 00, name: 'Old Hello', config:[{username: 'qwe', password: 'tyu'},{username: 'abc', password: 'xyz'}]}
title-update-error-response.json
{Error: 'not found', Message: 'The provided Book is not found.'}
book-record.feature
Feature: CRUD operation on the book records.
Background:
* def signIn = call read('classpath:login.feature')
* def accessToken = signIn.accessToken
* url baseUrl
Scenario: Change title of book in the single book-record.
* json ExpResObject = read('classpath:/book-records/title-update-response.json')
* json ReqObject = read('classpath:/book-records/title-update-request.json')
* call read('classpath:/book-records/update.feature') { Token: #(accessToken), ReqObj: #(ReqObject), ResObj: #(ExpResObject), StatusCode: 200 }
Scenario: Change title of book in the non-existing book-record.
* json ExpResObject = read('classpath:/book-records/title-update-error-request.json')
* json ReqObject = read('classpath:/book-records/title-update-error-response.json')
* call read('classpath:/book-records/update.feature') { Token: #(accessToken), ReqObj: #(ReqObject), ResObj: #(ExpResObject), StatusCode: 400 }
update.feature
Feature: Update the book record.
Scenario: Update single book-record.
Given path '/book-record'
And header Authorization = 'Bearer ' + __arg.Token
And header Content-Type = 'application/json'
And request __arg.ReqObj
When method put
Then status __arg.StatusCode
And response == __arg.ExpectedResponse
Actual API response for scenario: 1 is :
{name: 'New Hello', config:[{username: 'abc', password: 'xyz'},{username: 'qwe', password: 'tyu'}]}
Actual API response for scenario: 2 is :
{Error: 'not found', Message: 'The provided Book is not found.'}
Question: How should I validate the response in update.feature file since problem is if I make change s as using #^^config that will not works for scenario :2 and response == _arg.ExpectedResponse is not working for Scenario: 1?

This is classic over-engineering of tests. If someone has told you that "re-use" is needed for tests, please don't listen to that person.
You have two scenarios, one happy path and one negative path. I am providing how you should write the negative path here below, the rest is up to you.
Scenario: Change title of book in the non-existing book-record.
Given path 'book-record'
And header Authorization = 'Bearer ' + accessToken
And request {id: 00, name: 'Old Hello', config:[{username: 'qwe', password: 'tyu'},{username: 'abc', password: 'xyz'}]}
When method put
Then status 400
And response == {Error: 'not found', Message: 'The provided Book is not found.'}
See how clean it is ? There is no need for "extreme" re-use in tests. If you still insist that you want a super-generic re-usable feature file that will handle ALL your edge cases, you are just causing trouble for yourself. See how un-readable your existing tests have become !!
EDIT: Since I refer the question to others often as an example of how NOT to write tests, I wanted to make my point more clear and add a couple of links for reference.
Sometimes it is okay to "repeat yourself" in tests. Tests don't have to be DRY. Karate is a DSL that enables you to make HTTP calls or JSON manipulation in one or two lines. When you start attempting "re-use" like this, it actually leads to more harm than good. For example, you now need to look at multiple files to understand what your test is doing.
If you don't believe me, maybe you will believe the team at Google: https://testing.googleblog.com/2019/12/testing-on-toilet-tests-too-dry-make.html

Related

Firebase: Delete user analytics data - userdeletionRequests:upsert - GDPR

My question is, how can I delete a users analytics data from firebase using "Google User Deletion API" and its method: userdeletionRequests:upsert? This is important for me to fully fulfill GDPR.
I tried searching for this, but didn't a solution for using it in combination with "NodeJS" and "firebase-cloud-functions".
My biggest problem is, how I get the access, token, this is what I have for now:
const accessToken = (await admin.credential.applicationDefault().getAccessToken()).access_token;
return ky.post(constants.googleUserDeletionURL, {
body: JSON.stringify({
kind: "analytics#userDeletionRequest",
id: {
type: "USER_ID",
userId: uid,
},
propertyId: constants.googleUserDeletionPropertyId,
}),
headers: {
"Authorization": `Bearer ${accessToken}`,
},
}).catch((err) => {
functions.logger.log(`An Error happened trying to delete user-anayltics ${(err as Error).message}`);
});
But I always get An Error happened trying to delete user-anayltics Request failed with status code 403 Forbidden
Okay, after some painful and long days (literally took me like >20 hours), I've figured out how to achieve this request. Here is a step for step guide:
Step 1 - Needed Dependencies
To send a post-request to google, we need a http-client-library. I've choosen "ky", so we need to install it first with:
npm install ky
Furthermore, we need to create or OWN oAuth2 token, otherwise the post request will be denied with "error 403". To create our own oAuth token, we need another dependency:
npm install #googleapis/docs
Step 2 - Needed Google Property ID
With your request, Google needs to know which property-id / project you are targeting (where it should later delete the user-analytics-data). To get this property-id, we need to log in into GCP via Firebase (not the "real" GCP, the one via Firebase).
For this, go into your "console.firebase.google.com" → Select your project → Analytics Dashboard → "View more in Google Analytics (button at the right corner)"
Write "property-id" into the search field and then save it somewhere (we need it later)
Step 3 - Creating Client-Secret
The third step is to create a service-account, which we will later add into our functions-folder, in order to create our oAuthClient (don't worry, you will see what I mean to a later point)
To create your new service.json, log in to google cloud platform via "https://cloud.google.com/" and then follow the pictures:
FIRST:
SECOND:
THIRD:
FOURTH:
FIFTH
Step 4 - Download JSON
After we created our "oAuth-Deleter service-account", we need to manually download the needed JSON, so we can paste it into our functions-folder.
For this, select "oauth-deleter#your-domain.iam.gserviceaccount.com" under "Service Account"
Then click on "Keys" and "Add key", which will automagically download you a service-json (SELECT Key type → JSON → Create).
Step 5 - Paste JSON file into your functions-folder
To loosen up the mood a bit, here is an easy step. Paste the downloaded JSON-File into your functions-folder.
Step 6 - Grant Access to our new created oAuth-Delelter-Account
Creating the service-account and giving it access in the normal GCP is not enough for Google, so we also have to give it access in our Firebase project. For this, go back into your "GCP via Firebase (see Step 2)" → Click on Setting → "User Access for Account" → Click on the "plus"
Then click on "Add user" and write the email we copied before into the email field (the email from Step 3, Picture FOURTH "Service-Account ID). In our case, it is "oAuth-Deleter#your-domain.iam.gserviceaccount.com". Furthermore, it needs admin-access:
Step 6 - The code
Now, after these million unnecessary but necessary steps, we get to the final part. THE DAMN CODE. I've written this in typescript with "compilerOptions" → "module": "esnext", "target": "esnext". But I am sure that you are smart enough to change the code after completing this many steps :)
import admin from "firebase-admin";
import functions from "firebase-functions";
import ky from "ky";
import docs from "#googleapis/docs";
import { UserRecord } from "firebase-functions/v1/auth";
export const dbUserOnDeleted = functions.
.auth
.user()
.onDelete((user) => doOnDeletedUser(user))
----------------------------
export asnc function doOnDeletedUser/user: UserRecord) {
try {
const googleDeletionURL = "https://www.googleapis.com/analytics/v3/userDeletion/userDeletionRequests:upsert"
// Step 1: Paste propertyID: (See Step 2)
const copiedPropertyID = "12345678"
// Step 2: Create oAuthClient
const oAuthClient = new docs.auth.GoogleAuth({
keyFile: "NAME-OF-THE-FILE-YOU-COPIED-INTO-YOUR-FUNCTIONS-FOLDER",
scopes: ["https://www.googleapis.com/auth/analytics.user.deletion"]
});
// Step 3: Get user uid you want to delete from user-analytics
const uid = user.uid
// Step 4: Generate access token
// (this is the reason why we needed the 5 steps before this)
// yup, a single line of code
const accessToken = await oAuthClient.getAccessToken() as string;
// Step 5: Make the request to google and delete the user
return ky.post(googleDeletionURL, {
body: JSON.stringify({
kind: "analytics#userDeletionRequest",
id: {
type: "USER_ID",
userid: uid
},
propertyId: copiedPropertyID
}),
headers: {
"Authorization": "Bearer " + accessToken,
}
});
} catch (err) {
functions.logger.error(`Something bad happened, ${(err as Error).message)`
}
}
Afterthoughts
This was and probably will be my longest post at stack overflow forever. I have to say that it was a pain in the a** to get this thing to working. The amount of work and setup that is needed to simply delete a data from one endpoint is just ridiculous. Google, please fix.

URL parameters but ?mark is missing

I am new(trainee) to the web development and I just started studying its "language". I have to report every day to my boss but I have to use the correct terms. I have been reading about URL anatomy, in order to explain the fragments of this URL: https://www.c2ccertified.org/api_v2/search&username=rikotech&offset=50.
I see something that looks like query parameters to me: offset and username, but I don't see the question mark.
I don't know how to fragment(in correct terms) this part of the URL:
search&username=rikotech&offset=50
Are these query parameters? Because it seems to me that they serve that purpose - semantically. They filter the result from the API server.
------EDIT------
I found useful tool to use Node.js URL parser:
url = new URL(`https://www.c2ccertified.org/api_v2/search&username=rikotech&offset=50`)
console.log(url);
It did the job for me:
URL {
href:
'https://www.c2ccertified.org/api_v2/search&username=rikotech&offset=50',
origin: 'https://www.c2ccertified.org',
protocol: 'https:',
username: '',
password: '',
host: 'www.c2ccertified.org',
hostname: 'www.c2ccertified.org',
port: '',
pathname: '/api_v2/search&username=rikotech&offset=50',
search: '',
searchParams: URLSearchParams {},
hash: '' }
If there's no "?", then there's no query, thus no query arguments. See https://www.greenbytes.de/tech/webdav/rfc3986.html#components.

How do I delete user analytics data from Firebase using userDeletionRequests:upsert?

Problem Description
My Android app collects data via Google Analytics for Firebase. For privacy reasons, users must be able to wipe their data off the Firebase servers, should they choose to do so.
The app requests a deletion by forwarding its Firebase APP_INSTANCE_ID to my own server. This server has been prepared in advance with credentials, from my personal Google account (via oauth2), for managing the Firebase project. The server authenticates with www.googleapis.com, and, using the supplied APP_INSTANCE_ID, invokes the upsert.
As noted by the documentation, the generic Google Analytics API is appropriate for this task.
After some initial trouble (b/c I didn't have the correct auth scope, and the Analytics API wasn't properly enabled), googleapis.com now returns HTTP 200 for each upsert request. (As an aside, even if you supply a bogus APP_INSTANCE_ID, it returns 200.)
Here is a sample response from the upsert, which shows nothing amiss:
{ kind: 'analytics#userDeletionRequest',
id:
{ type: 'APP_INSTANCE_ID',
userId: (REDACTED 32-char hexidecimal string) },
firebaseProjectId: (REDACTED),
deletionRequestTime: '2018-08-28T12:46:30.874Z' }
I know the firebaseProjectId is correct, because if I alter it, I get an error. I have verified that the APP_INSTANCE_ID is correct, and stable up until the moment it is reset with resetAnalyticsData().
Test Procedure
To test the deletions, I populated Firebase with several custom events, using the procedure below (Nexus 5X emulator, no Google Play, no Google accounts configured, but that shouldn't make any difference):
Install the app
Fire off some custom events (FirebaseAnalytics.logEvent)
Observe those events appear on the Firebase console
(About a minute later:) Make the upsert call, observe HTTP 200, and note the "deletionRequestTime"
Immediately call FirebaseAnalytics.resetAnalyticsData (to clear any event data cached on the device)
Uninstall the app
Rinse & repeat 7 or 8 times
However, even 24 hours later, 100% of the Firebase events are still present in the events table. No discernable state change has taken place on the Firebase server as a result of the upserts.
Question
So, what am I doing wrong? how do I successfully delete user data from Google Analytics for Firebase?
EDIT
Here's the code I'm using to make a request (from node.js):
const request = require( 'request' );
...
_deletePersonalData( data )
{
return new Promise( (resolve, reject) => {
request.post({
url: 'https://www.googleapis.com/analytics/v3/userDeletion/userDeletionRequests:upsert',
body: {
kind: 'analytics#userDeletionRequest',
id: {
type: 'APP_INSTANCE_ID',
userId: data.firebaseAppInstanceId
},
firebaseProjectId: (REDACTED)
},
headers: {
Authorization: 'Bearer ' + iap.getCurAccessToken()
},
json: true
}, (err, res, body) => {
console.log( 'user-deletion POST complete' );
console.log( 'Error ' + err );
console.log( 'Body ', body );
if( err )
{
reject( err );
return;
}
if( body.error )
{
reject( new Error( 'The Google service returned an error: ' + body.error.message + ' (' + body.error.code + ')' ) );
return;
}
resolve({ deletionRequestTime: body.deletionRequestTime });
});
});
}
Here's a sample request body:
{
kind: 'analytics#userDeletionRequest',
id: {
type: 'APP_INSTANCE_ID',
userId: (REDACTED 32-char hexidecimal string)
},
firebaseProjectId: (REDACTED)
}
And here's the console output for that same request (same userId and everything):
user-deletion POST complete
Error: null
Body: { kind: 'analytics#userDeletionRequest',
id:
{ type: 'APP_INSTANCE_ID',
userId: (REDACTED 32-char hexidecimal string) },
firebaseProjectId: (REDACTED),
deletionRequestTime: '2018-08-29T17:32:06.949Z' }
Firebase support just got back to me, and I quote:
Upsert method deletes any individual user data we have logged, but aggregate metrics are not recomputed. This means that you might not see any changes in the events tab in your Analytics console.
So, basically my mistake was expecting the events to disappear from the console.
This, of course, raises the question of how one determines that the API is actually working... but maybe the HTTP 200 is enough.

Firebase Firestore comment tree architecture

I'm trying to implement a Reddit/HackerNews style tree of comments as part of a project, and am trying out Firestore as a database solution. However, I'm unsure as to the correct design reading through the docs. In a SQL database I would use numeric keys like:
0
1.0
1.1
1.1.1
0.0
to represent my tree. However, numeric keys like that seem to be a Firebase antipattern. The other route is using an actual tree in the json where a post is represented like:
{
uid: 'A0000',
content: 'foo',
children: [
{uid:..., content:..., children: []}]
}
but supposedly deep trees are bad in Firestore. As I understand it the reason deep trees are bad is that you have to fetch the whole thing, but in my case I'm not sure if that's a problem. A client fetching a post would fetch the root content node and the first 20 or so child trees. That could be a pretty big fetch, but not insanely so.
Does anyone know of a good standard way to implement this kind of structure?
Extra: Here is the more verbose expression of what the structure should look like once the client processes it.
{
uid: 0,
title: 'Check out this cat!',
body: 'It\'s pretty cute! This **text** is [markdown](link), so it can have ' +
'links and *stuff*. Yay!',
poster: {
uid: 0,
name: 'VivaLaPanda',
aviUrl: 'badlink',
},
posted: '2018-03-28',
children: [{
uid: 0,
body: 'This is a comment, it\'s angry!',
poster: {
uid: 0,
name: 'VivaLaPanda',
aviUrl: 'badlink',
},
posted: '2018-03-20',
children: [{
uid: 0,
body: 'This is a comment, it\'s neutral!',
poster: {
uid: 0,
name: 'Steve',
aviUrl: 'badlink',
},
posted: '2018-03-20',
children: [{
uid: 0,
body: 'This is a comment, it\'s neutral!',
poster: {
uid: 0,
name: 'Craig',
aviUrl: 'badlink',
},
posted: '2018-04-10',
children: []
}, ]
}, ]
},
{
uid: 0,
body: 'This is a comment, it\'s happy!',
poster: {
uid: 0,
name: 'Craig',
aviUrl: 'badlink',
},
posted: '2018-03-28',
children: []
},
]
};
Edit:
While I've marked this as answered because there is an answer, I'm still really interested in seeing something more elegant/efficient.
Edit2:
For posterity: I ended up deciding that any Firebase solution was hopelessly convoluted and just used DGraph for the data, with Firebase sitting in front for Auth.
This is tough since the structure you have is naturally recursive. The obvious options are each comment is a new document in a collection and each reply is a single document in the same collection.
Each comment as a new document could work something like this. Each comment has a "postId" attribute which dictates which post it belongs to. Some comments, those which are replies to other comments, have a "replyToId". These two attribute in conjunction allow your client app to:
Get the top level comments (look for comments with the correct postId and which don't have replyToId). Top level comments allows you to limit the size of payloads if you need to worry about that in the future.
Get all comments (look for comments with the correct postId only). If you don't care about payload sizes you can get everything and figure out the tree structure on the client.
Get replies to a particular comment if you want "see replies" YouTube style comments interaction (looks for comments which have a particular replyToId). This works well in conjunction with 1. for limiting payload sizes.
But the logic here is obviously complex.
Your children approach could be really messy if there are a lot of people commenting on eachother. A nicer approach would be the following structure for every comment:
// single comment
postUid // <- random generated by firebase
{
postedBy: userUid
postedTime: timestamp
postIsChildOfUid: postUid // <- reference to an other post (optional if the comment didn't respond to another comment(top-level comment))
}
This doesn't even require nesting at all :). You can generate easily now a comment tree with this approach, but this has to be client side. But that should be easy!

Uploading of image to WordPress through Python's requests

In order to validate the installation of WordPress instances, we are writing Python unit tests. One of the test should perform the following action: upload an image to WordPress.
In order to do that, I am using the Requests library.
When I inspect the form within /wp-admin/media-new.php page through Firebug (form information, I get the following information):
Form
Id: file-form
Name
Method: post
Action: http://localhost:8000/wp-admin/media-new.php
Elements
id: plupload-browse-button
type: button
value: Select Files
id:async-upload
name: async-upload
type: file
label: Upload
id:html-upload
name: html-upload
type: submit
value: Upload
id: post_id
name: post_id
type: hidden
value: 0
id: _wpnonce
name: _wpnonce
type: hidden
value: c0fc3b80bb
id: file-form
name: _wp_http_referer
type: hidden
value: /wp-admin/media-new.php
I believe that the _wpnonce is a unique value generated for each session. Therefore, before trying to upload the file, I get the media-new.php page and grab the _wpnonce in the form (hence the variable in my code).
My code is the following:
with open('1.jpg', 'rb') as f:
upload_data = {'post_id': '0',
'_wp_http_referer': '/wp-admin/media-new.php',
'_wpnonce': wp_nonce,
'action': 'upload_attachement',
'name': '1.jpg',
'async-upload': f,
'html-upload': 'Upload'}
upload_result = session.post('http://localhost:8000/wp-admin/media-new.php', upload_data)
The code runs fine and the upload_result.status_code equals 200.
However, the image never shows up in the media gallery of WordPress.
I believe this a simple error, but I can't figure out what I'm missing.
Thanks in advance for the help.
If you want to post files you should use the files parameter. Also the '_wpnonce' value is not enough to get authenticated, you need to have cookies.
url = 'http://localhost:8000/wp-admin/media-new.php'
data = {
'post_id': '0',
'_wp_http_referer': '/wp-admin/media-new.php',
'_wpnonce': wp_nonce,
'action': 'upload_attachement',
'html-upload': 'Upload'
}
files = {'async-upload':('1.jpg', open('1.jpg', 'rb'))}
headers = {'Cookie': my_cookies}
upload_result = session.post(url, data=data, files=files, headers=headers)
I'm assuming that you have acquired valid cookies from your browser. If you want to get authenticated with requests check my answer to this post: login-wordpress-with-requests

Resources