I'm trying to collect data from a website using a bot in order to simulate a real web traffic, is there anyway to collect a stream of data (apache log lines).
Thanks guys, that would be really helpful.
Related
Can we connect a web application to ONA ODK forms ?
I need to replace a system where employees goes to field, collect information, and then sync to database when they come back.
What we need is to find a solution, by creating odk surveys, let the employee collect data, and then sync directly to ona and mysql database. And I need to give them the ability to read and edit data directly form the forms into database.
I know the idea is a little bit strange.
You should ask this question in ODK Community and as you want a solution for sending data that can be done by the cloud-based server like google cloud and for editing data. It's not a good practice to edit data in a raw dataset. You can download data from the server and then you can make a change in your computer.
Even if you want to give them access to the server so they can log-in and make a change on the server, you can create a username and password so they can update their records.
Does someone knows a way of sending an HTTP post to a URL from a VG-IDMS program?
We have a mainframe database with a few programs written on it and we have so far successfully created functionality for it to work as a kinda web service provider, responding to HTTP get requests with XML data. That way we've been able to create external (and modernized) apps that requests data from it as if would be a modern server.
Now we'd like to make the opposite. We want certain VG programs to create and send HTTP post requests to an external URL (most likely JSON formatted although not important). The objective is to create some sort of notification so that when certain events happen on the mainframe (say data got updated) a VG program would notify an external web service (or web api) of it.
I've been trying to find documentation all over but failed so far.
EDIT: Replacing the old mainframe is not an option for now
I just started working on Google Analytics stuff and I'm pretty new to this. I am now granted access to GA account of my Organization's marketing Website for several European countries(single login).
My requirement is to copy different European countries GA data into a single table structure in SQL server. Wondering if anyone of you have done this before? Any suggestions are highly appreciated.
As already written earlier, there are several ways of doing this. I prefer to integrate Google Analytics and SQL Server with no coding, using Skyvia tool: Google Analytics and SQL Server Integration. It allows me to create a copy of Google Analytics report data in SQL Server and keep it up-to-date with little to no configuration efforts. I don’t even need to prepare the schema — Skyvia can automatically create a table for report data. You can load 10000 records per month for free — this is enough for me.
There is a number of ways of doing this. Google Analytics does have the ability to export data as CSV but its going to be hard to match up the data properly.
If you are up for a bit of programming. start with the Google Analytics API it will allow you to extract data from Google analytics and insert it where you like. You can use any programming language that is capable of preforming a HTTP Post and HTTP Get. However i recommend looking into one of Googles client libraries.
If you have the ability to use SSIS to you can use Targit Google Analytics SSIS its a custom connection manager and data reader for extracting data from Google analytics it is free to use. Note: Full disclosure I am the lead developer on that project.
I need to build a server to an application who's client side is written in Lua using Corona SDK (application for mobile smartphones and tablets), and server side in .NET.
I have thousands of clients who I need to push data to in a form of XML file or JSON array, every few hours or daily. (Not notifications. Data that needs to reach the application directly and trigger certain actions).
Is there a way to push the data without WCF?
The only other way I found was SignalR, but my client isn't a web browser so I ruled it out.
In my searches I found in stackoverflow a question with a recommendation to push data using 0MQ (http://zeromq.org/), I just found it hard to understand if it fits my needs, can I queue so many clients in one server and delay the answer a few hours or days?
Other recommendations would be greatly appreciated, thank you.
Say, I'm developing a Windows (if OS is important) application that will be available to download for free and I would like then to collect some usage statistics. In the easiest case - count of application launches. It looks superfluous to maintain a server (e.g. VDS) just for this.
I've been thinking to use Google Analytics for this (manually send requests to GA server). This will probably work, but it is not GA designed for - the idea looks like a hack.
What are the options here?
I don't think this is a hack. It's all just data about user interaction. There is little logical difference between opening a desktop app and clicking a button vs opening a web page and following a link. Both are measurable user actions you can track, aggregate and put on graphs.
In fact, Google provides a lower level HTTP based "Measurement Protocol" that is intended for exactly that.
https://developers.google.com/analytics/devguides/collection/protocol/v1/
From the overview:
The Google Analytics Measurement Protocol allows developers to make
HTTP requests to send raw user interaction data directly to Google
Analytics servers. This allows developers to measure how users
interact with their business from almost any environment
Just put an HTTP request with the correct parameters in your application launch or button click code and it will collect the data. Any data you want to collect.
In other answers to this question there are suggestions like making web services or storing the data locally but why reinvent the wheel? Google Analytics already provides the collecting and reporting tools and it seems like a good solution.