which servies to use for periodically load data from multiple data sources, aggregate and provide fast search? - airflow

Please propose a solution design for my case. The data comes from various sources, some from api, some from csv. A user will search using filters.
Ex: Product data (source 1) and Product Reviews ( Source 2). A user will search for a product with its name and rating.
Considerations:
When product reviews will keep on changing. So, the search should reflect that information.
In future, some more sources will get added and additional filters will get added. Ultimately, only products will be shown to the users.
Product price will change. The search results should give updated information.
The search should be very fast.
I looked into using apache airflow, mongodb and elastic search. However, i felt i am complicating the solution.

Related

Multi Store with Subdomains

I want to create multiple online shops for selling merchandise products for companies. The products are basically identical but should be personalized in dependence of the company I am building the shop for. Because I do not want to build a new shop every time a new company joins the program I am looking for something like that:
www.myshop.com : One shop with the underlying product database and checkout system - not showing any products, just as a parent structure
www.company1.myshop.com : A slightly personalized shop where only a selection of the product catalogue is available
www.company2.myshop.com : A slightly personalized shop where a different selection of the product catalogue is available
Do you get it?
Does anybody know a tool for that?
Thanks in advance!
I already looked into WooCommerce, Shopify and even WiX. As far as I understood what I am looking for is not supported.
Since your example is based off of subdomains, you can choose to assign a Shopify store to each subdomain. Each store feeds from your inventory and accounting, giving your customers the illusion of a custom experience. Or you can just simplify your life, have one store, and assign your customers to view collections specific to them. That is the smart move. You may not like that, but it would work a peach for you. You just tag customers to see their specific collections, of products specific to those collections. Simple.
I can also think of a dozen other ways to pull this off with Shopify, but that is me, not you. For an opinion question like this, SO is not the right place to ask these kinds of questions, but I answered anyway. Your mileage may vary of course.

How to analyze multiple query parameters in Google Analytics

I'm setting up Google Analytics for a website where a user can find an event to attend (concerts, plays, etc.). The results can be filtered by 5 different parameters.
So, unfiltered results would look like: example.com/event-finder/
And filtered results showing concerts in January or February would look like: example.com/event-finder?type=concert&month=jan,feb
I'm struggling to figure out the best way to use the query parameters in Google Analytics to analyze filtering behavior.
Example questions I'd want to be able to pull answers for:
What percentage of results were filtered by type?
What percentage of results were also sorted by month?
What is the most common type filtered by?
I have full access to both Google Analytics and Tag Manager but I suspect I shouldn't do this with events or custom dimensions and that there's got to be a way to use the query parameters to do this in a clean way.
I've tried to use a new view and site search to group the types of filters. Seems like it could work, but seems hacky and limited.
I've considered pushing those values into custom dimensions, but that too seems like overkill.
I've considered pulling content reports into Google Sheets and sorting through things there, but I'm 1) not entirely sure how I'd do that and 2) suspect there may be an easier approach.
Let me know if you have any questions or need more clarification. Thanks!
Have you tried to use "category parameters" when configuring site search (admin -> view settings)? You could set the "type" as a category parameter. You can also enter multiple parameters in there.
Check this screenshot of site search configuration

Best Way to Pass variable to Woocommerce Product?

Here is the setup.
Customer fills out form of information for product specifications. This creates an entry in the db with a unique detailsID.
They then are taken to a product page, with many variables. They select the variable product and checkout.
Now in the db they have an order as well as a separate table with the details of their request.
What is the best way to connect the product they bought with the detailsID?
I would like to basically pass the detailsID to the product page, then add this detailsID to the product. Or add the orderID back into the details table.
I keep going around whether to add this as meta data, or use a session, or just do post order logic to add the order ID back into the details table.
I really just need to link the two once the order completes but am not sure the best method to do this. I am pretty new to Woocommerce so don't know a lot of the hooks yet.
Thank you so much for your suggestions!
After a lot of research and experimenting here is the method I have decided on. This seems to add the information in a variety of places to make it easy to track orders and to insure that the customer is getting the customized product they want.
add custom data to woocommerce
I am using this method very close to how it is spelled out in this great article. This was really helpful for me being a woocommerce newb.

Is it possible to track actual affiliate sales in Google analytics

I have a website that I built and am getting sales everyday now via "affiliate window" I was curious if it was possible to track the commission in analytics so I am able to see what pages are performing the best?
Thanks
You should look into purpose-built affiliate trackers. Voluum, Thrive, Adsbridge etc. CharlesNgo.com has a lot of info about these.
You can also use these trackers to dynamically insert user data into your landing pages, and they are a lot easier than GA to use. Example: https://charlesngo.com/how-to-insert-user-data-into-landing-pages-using-voluum-tokens/
After a clarifying comment: Yes, you can do this, although with some caveats (one of them that it might not be worth the effort unless you make a lot from your affiliate pages).
You can do data imports, and more specifically you can add data to urls (there are different kinds of data import and "content data" is one of them).
You need a dataset that contains a "key" field that is used to match external data to GA data and one or more fields with values you want to import.
Imported data is always dimensions, i.e. categorical data (that is per Documentation The interface allow to to select custom metrics, but I have not yet tried this and cannot give any guidance on how that would work). So if you try to import revenue you have the problem that new entries will not be added to existing entries, they will replace them
What I would probably do is to sort the data from the "affiliate window" into three categories (low, medium, high), and then prepare a csv file with the urls as first column and the categories as second column.
Then create a custom dimension with a name of i.e. "performance" (else you'd have to overwrite an existing dimension in GA and you probably not want that).
Both custom dimensions and data imports are created on a property level (however you can apply an import to a specific view and I would urge you to test this on a test view first). So in the property settings go to data import, new, content data. Key will be "page", Imported data will be the custom dimension you just created. Check "overwrite hit data" (else the data will not change after the first import, however note that this might make comparisons between different timeframes difficult).
Download the "schema" file (simply a template for your csv upload file in which you insert your data). Click finish.
Next go to data imports, "manage uploads" and upload the file. Processing will take a day or so (errors, if any, you will see shortly after the upload).
Then go to your content reports, select your custom dimension as secondary dimension and you will if the url in question was a low, medium or high performer.
You can automate this via the GA API, bit that's a bit beyond an SO answer.
---- (old answer)---
This is actually what campaign parameters are for. Your affiliate links should be tagged with campaign parameters, e.g.
http://example.com/?utm_medium=affiliate&utm_source=[[Affliate
Name]]&utm_campaign=[[Campaign Name]]
The things in the angled brackets are placeholders, you would replace those with the name of your affiliate and the name of your campaign.
Now you can look into the aquisition reports and group by source (values for all affiliate links), by source (breakdown by affiliate name) or campaigns, or combinations thereof.
However tracking the commission you pay out via Google Analytics is probably not a good idea, at least from the point of view of your affiliates - JavaScript based tracking is not necessarily accurate enough to track billable services (some people have js disabled or opted out of GA or use adblockers that block tracking etc).
Yes it's possible. You can use sub-id's and fill them with unique visitor data (Google Analytics Used ID). Once an affiliate sale is tracked in an affiliate network, you'll get the sub-id that generated the sale. You can now push that sale to google analytics and let google analytics match the visitor data with their data, showing you the full visitor reports (including landingpage info etc.). You need developer skills to get this working with your affiliate networks but there are complete easy to use tools that does the trick for you such as Ivanhoe.io and Coincrack.

Wordpress : data exchange betweeen two plugins

I want to develop a website where teachers create exams and students pay money in order to pass these exams, so it's an e-commerce website where products are quiz.
The exams will be managed with a quiz plugin : i'll use Watu.
I still have no idea what e-commerce plugin I'll use.
What I want is to make exams as products so students can add them to cart, but I dont't see how can I do it : the e-commerce plugin manages its own products !!! Is there an easy way to do it ?
This is my idea: alter the quiz module so when an exam is created the script insert a row in the e-commerce table(s), so when I go to the e-commerce plugin I'll find a new product! But I think this is not a clean way !!
You might consider setting up woo-commerce setting a link as a software product that will be delivered on checkout.
You could build a quiz answer table in sql, deliver a generic quiz (via a link sold in the store)
the quiz table could have a field for quiz_id and you could have their answers recorded as a text field.
so the link would be delivered digitally thru the store, your answers 1-100 a-d might look like this:
userid: xxxxx
autodate
test: 1
Answer:abadddaeababadddaeababadddaeababadddaeababadddaeababadddaeab
so one entry would contain all the answers on the test in question in one field.
once committed you could return a result based on a difference function from a pattern mask.
how many wrong answers in the simple query.
You then can store the answers and draw data based on what particular answers they got wrong.
they can also retake the same test at a later date and you can see each time how they did.
advantages of using a single text field means that you can use a single compare to pull the data and it's short to store.
with a membership plugin. Give roles afther payments. Diferent roles for diferent levels of exam. It´s easy than make conexion with database.

Resources