For an university project (Big Data lecture), I’d like to analyze auctions on eBay. I wasn’t able to find reliable information so far whether it’s possible to get all current auctions on eBay via their API or not. I only need the auction title and the current price and I am aware that this is a huge load of data, but I’m just curios.
I don't think it's possible, in part because of the huge amount of data, and perhaps also because I don't think eBay wants people downloading data en masse like that. Doing so might allow people to do data mining and market research from a vantage point that is too publicly revealing for them.
If you're willing to settle for a large segment of data, look into eBay's Large Merchant Services and their LMS API.
For your research project, you should be able to make sense of an even smaller subset of data by just pulling from eBay's Finding API in a few automated large chunks.
Related
I am working on a research project based on the six degrees of separation theory (based on the works of Stanley Milgram 1967 and later, Duncan Watts 2001). I just wanted to plot the connections of every user part of the study.
I wanted to show a users connections list, from LinkedIn. I read that there was something like a peoples API which could be used, but I think it is deprecated now. Is there any way of getting a users list of connections, even if we take the users permission?
I also read that there is something like a LinkedIn partners program, I cannot find much information about how I can apply for it.
I run a monthly report which tracks session views by region, most popular knowledge articles, deflection rates, most popular product pages, software download stats, etc.
We have a new ELT member who is keen to get into the numbers around our contact centre. As I only look after the support site I need only concern myself with putting together a report which outlines what I feel will be useful information around web traffic. I want the report to be brief, and to highlight 4-5 key metrics.
Please can I have some suggestions for data you think would be useful given the target audience?
So far I am considering:
Deflection rates
Bounce rates.
Time on page
Most popular software downloads.
Global session views year to date.
Any help would be really appreciated. Thanks!
I think those metrics are great. Ideally, the value in the data comes from slicing your metrics with a dimension, ie pivoting. For example, bounce rate as an average means little whereas bounce rate by Content Group or Device Category would be more interesting.
Speaking of Device Category, consider completely isolating the metrics for Mobile vs Desktop+Tablet. Those experiences are so drastically different you'd be doing a disservice to average those metrics together.
Lastly, I'd say this new ETL member should get their own access to GA and learn how to pull the data need. GA now offers machine learning insights that quickly surface relevant drivers in metrics; a static approach to KPI reporting is becoming increasingly obsolete.
I have participated in a Hackathon in my city, and the traffic department made public a dataset with more than 250 thousand traffic accident datapoints, each one containing Latitude, Longitude, type of accident, vehicles involved, etc.
I made a test to display the data using Google Maps API and Google Fusion Tables, but the usage limits were quickly reached with the first two years of a total of 13 years of records.
The data for two years can be displayed and filtered here.
So my question is:
Which free online services could I use in order to interactively display and filter 250 thousand such datapoints as map layers?
It is important that the service be free, because we are volunteering our time for non-profit public good. Currently our City Hall is implementing an API, but it is not ready yet, and it would be useful to present them some popularly well-accepted use-cases to make some political pressure for further API development with THEIR server (specially remotely querying a database instead of crawling a bunch of .csv files as it is now...)
An alternative would be to put everything in GitHub and load the whole dataset client-side to be manipulated with D3.js for example, but that seems very inefficient either for the client/user as for the server.
Thanks for reading, and feel free to re-tag if needed.
You need Google Maps API for Business to achieve what you want, but it costs a lot of money.
However, in some cases, you can get this Business Licence if you work for non-profit organization. I can't find the exact rules to be eligible for this free licence. I tried googled them but I can't find anything. I only find this link, just take a look if it can answer your problem.
You should be able to do that with Google Fusion Tables. The limit is 100,000 points per table, but you can overlay 5 layers onto a single map so in effect you can reach 500,000 points. I implemented the website below and have run it with over 200,000 points.
http://www.skyscan.co.uk/mapsearch.html
We have been asked to increase the performance of a clients site search. Before we start we would like to set benchmarks. I have asked the client if they are comfortable with enabling unanimous data sharing so we have access to industry benchmarks as I don't have control over this setting: http://support.google.com/analytics/bin/answer.py?hl=en&answer=1011397 however it sounds like things have changed in the google analytics camp and these reports are only available via a newsletter now? Is this true?
Also, will these reports give me industry standards to compare my clients current search performance against? Or is there another service that has these baseline standards available?
Here's an example of the data we are interested in. This is our clients current search performance:
Visits with Search: 772
Total Unique Searches: 1,093
Results Page Views/Search: 1.36
% Search Exits: 56.45%
% Search Refinements: 24.78%
Time after Search: 00:01:40
Search Depth: 0.59
I work at large ecommerce site, and I asked our AdWords rep about this, having recently wanted access to this kind of data myself.
He said that benchmarking was removed 3/15/11, at which point they were experimenting with a monthly newsletter format to deliver the same kind of data.
From what I've seen they may have done one newsletter before (quietly) retiring it completely. I never saw the newsletter, but I think I remember reading reports of people who did receive one.
Disappointing to know they had access to all that data, but pulled the plug on the program. I wonder if they killed it due to data integrity concerns--they can't guarantee correct tracking-code installations on all these sites opting in, so what is the data worth if it's of questionable quality. iono... just a total guess.
We used to use coremetrics here, and they had an opt-in benchmarking program. So if you know any other webmasters using Coremetrics, you could probably ask them to pull some benchmarking info.
We were able to get some benchmarking data from fireclick.com, but none of it (that I've seen anyways) covers on-site search. Mainly just top line metrics. :-/
So the search for benchmark data continues...
The company I work for has just purchased 4 32" LCD screens to be mounted at the front of the office for demonstration purposes. Whilst we are not demonstrating (most of the time), the screens are to be used as development information screens for the whole team.
What information would people recommend displaying to be most useful to the team? Our focus is on hosted business web-apps but I am interested in what other teams doing other types of development find useful too. Pointers on how to gather the displayed information would be useful also.
Information about your continuous integration status.
Major Development Milestones that have been hit in the last week
Releases within the last month (including a short description why this release is awesome)
Use it as motivational board. The achievements of software development are seldom communicated well enough.
Since you're hosting apps for your customers, server and network status information would probably be useful.
Heck, why not create a "chat room" for the dev team to discuss issues and post a streaming version of that as well?
Schedule information, Scrum notes from that morning, a gantt chart...the possibilities abound.
Outstanding bugcount, sorted by priority and severity. You can likely get this from your bugtracking tool programmatically.
Depending on your process management
system, possibly a list of feature
requests and the percentage complete
on each of them. Again, you can probably get this programmatically from your process management / time tracking tool.
Time spent in the current development
cycle, and time remaining. Again, this should be available from your process / management / time tracking tool. You may want to use this data with your bugcounts as well to give a bugs / day fix rate.
If you're a public company with a
profit-sharing plan (i.e. stock or
options), the current price of the
stock (this can be surprisingly
strongly motivating). You can get stock data from several sources online programmatically (although a small delay may be injected unless you're paying for the service).
The movie 'Office Space'
Weather radar from intellicast.com
Latest Checkin.
Number of checkins per day
Number of customers that use software
Metrics on Bugs found/fixed and the ratio.
One screen could be an aggregated RSS feed of development topics pulled from sites such as Stack Overflow (or even Coding Horror). Not sure what your goal for these screens is, but I could see it useful to me if you had a feed with topics specific to your development team headlined. If I were there, I'd glimpse them, maybe catch an interesting thread, and go learn something. Funnel a bunch of keywords and tags through a Yahoo Pipe and dump it to the screen.
That's if they are more "informal and informational."
I think most popular pages from your webapp(s) would be a fun/interesting thing to show on a big monitor up front.
Another would be a live feed of your error reporting.
We have one monitor showing all meetings for the day, with start-end, subject, and room. I find this helpful, not only for my orientation, but also to see what other people do at our company.
xkcd, bunny, dilbert and savage chickens :-)