i am using bika lims to develop a lims . I am trying to maintain a universal lims say bikaA wherein I can store all possible information(list of all types of tests, their costs, containers) which can be used later for developing similiar subsystems bikaB, bikaC etc which will contain only subsets of the information stored in bikaA depending on the needs of the specific clients.
Basically I want to extract information from one universal lims to sub lims which are the actual end products.what I want to ask is:
1.How can I extract data from universal lims- bikaA and convert this data to different formats like csv.
2. how can this data be imported into child lims .
Bika LIMS uses spreadsheets containing setup data (example here) which you can use to pre-configure the things you mentioned and quite a few more. You can edit this sheet and re-import into each new LIMS instance. You can also import subsections of this spreadsheet into an existing bika.lims site. The importer does have a few rough edges (most notably, the error messages during import are not very easy to read), but it works quite well for us.
Right now there is no good solution to do exactly what you want, but there are export and import scripts located here. They are simple command line scripts, and they don't use any Plone migration tools and addons. They could serve as a starting point. It's planned to clean these up and include them in Bika LIMS as soon as there is enough interest or time.
As Luca Fabbri already said, you can use the Bika mailing lists (or slack channel) if you have specific questions - you might get a faster response over there.
hope this helps
Related
I'm not a web developer, so please bear me.
https://www.etoro.com/people/hyjbrighter/chart
I know that there are several libraries to plot graph in Javascript but how can I check if a specific page is using highchart or another competitor?
I expect to find some kind of Json in the source code but how can I find it?
The trick is to open the Network tab of Dev Tools, reload the page, and search for the piece of data that you want to scrape. Here I saw a number is 21361.15, I searched for it and detected the JSON file is from https://www.etoro.com/sapi/userstats/CopySim/Username/hyjbrighter/OneYearAgo?callback=angular.callbacks._0&client_request_id=2ce991a6-0943-4111-abd3-6906ca92e45c.
But you need to clear the parameters in this situation to actually get the proper information.
I don't know which language you use, if you use Python, here is the code:
import requests
import pandas
data = requests.get("https://www.etoro.com/sapi/userstats/CopySim/Username/hyjbrighter/OneYearAgo").json()['simulation']['oneYearAgo']['chart']
data = pandas.DataFrame(data)
print(data)
Output:
If you use R, use jsonlite package.
I am trying to automate a process that is currently done mainly with excel files. These files have been used for a while and are customized just how the user likes them. I am turning this into a data driven VB NEt application and now and at the task of configuring all the computed columns to do the equations the user's excel spread sheets are doing currently.
The main ones needed that I can't find information on are STANDARDIZE, PERCENTRANK and STDEVA (atleast for computed columns- I have seen STEVA used in select queries)
Excuse me if there is documentation on this I can refer to, I searched google and stackoverflow and wasn't able to find anything. If you could point me to any documentation like this that might exist- that would be a huge help!
I am trying to use broadleafcommerce and customize it.On study i found it uses Apache Solr . However, i am already handy with
ElasticSearch as i am currently using ElasticSearch only in my workplace. so, i'm curious as if i can replace that customizable code of broadleafcommerce with ElasticSearch. If it is possible, i also want to know how long will it take or what will be its difficulty level ?
Thanks in advance !
The product is open source, you can have a look at the code yourself. Here is the package that would need to be made solr independent. As far as I see there are quite some dependencies on Solr now, but maybe you can give it a shot and contribute it back. In the end that's the power of open source.
I can't tell exactly how much work that would be since I don't know the product and what it does with the data. The solr schema would need to be translated to the related elasticsearch mapping, then the indexer will need to be converted in order to push data to elasticsearch (otherwise if technically doable you could write a river that imports data in elasticsearch from the framework itself). Last step is to convert the search code together with the facets, highlighting etc.
Maybe you (or the people behind the project) might want to have a look at spring data which has now a community driven spring-data-solr project and an unofficial elasticsearch implementation too.
I am uploading several files to Alfresco repsitory via webdav. The batch process works fine, but after the upload, all dates in the repository are changed to current date.
How can I make it keep and show the original file dates (creation and modified) ?
Thanks.
You can leverage metadata extractors. The main purpose is to extract metadata from binary files during upload. There are lots of built-in metadata extractors, just look for implementers of interface org.alfresco.repo.content.metadata.MetadataExtracter. There are different extractors that can extract creation date and set it as cm:created on Alfresco node.
You can enable metadata extraction by applying it as a rule on a space, look for action named Extract Common Metadata in the actions drop-down-box while creating the rule.
I don't believe it's possible without the importing code explicitly turning off the default behaviour of the "cm:auditable" policy, and I suspect the WebDAV driver doesn't do this (since it has no way of knowing whether that's appropriate or not - there are cases where forcing the creation and modification dates to today is the correct thing to do).
This behaviour is discussed in some detail here - it might be worth evaluating whether the bulk filesystem import tool is a more appropriate way to import the content into Alfresco, particularly since it can preserve the creation and modification dates if you tell it to (i.e. by specifying the values of those properties).
I know this isn't programming related, but I hope some feedback which helps me out the misery.
We've actually lots of and different data from our web applications, dating years back.
For example, we've
Apache logfiles
Daily statistics files from our tracking software (CSV)
Another daily statistics from nation-wide rankings for advertisement (CSV)
.. and I can probably produce new data from other sources, too.
Some of the data records started in 2005, some in 2006, etc. However at some point in time we start to have data of all of them.
What's I'm drea^H^H^H^Hsearching for is an application to understand all the data, lets me load them, compare individual data sets and timelines (graphically), compare different data sets within the same time span, allow me to filter (especially the Apache logfile); and of course this all should be interactively.
Just the BZ2 compressed Apache logfiles are already 21GB in total, growing weekly.
I've had no real success with things like awstats, Nihu Web Log Analyzer or similar tools. They can just produce statical information, but I would need to interactive query the information, apply filters, lay over other datas, etc.
I've also tried data mining tools in hope they can help me but didn't really success in using them (i.e. they're over my head), e.g. RapidMiner.
Just to make it sure: it can be a commercial application. But yet have to find something which is really useful.
Somehow I get the impression I'm searching for something which does not exist or I've the wrong approach. Any hints are very welcome.
Update:
In the end I it was a mixture of the following things:
wrote bash and PHP scripts to parse and managing parsing the log files, including lots of filtering capabilities
generated plain old CSV file to read into Excel. I'm lucky to use Excel 2007 and it's graphical capabilities, albeit still working on a fixed set of data, helped a lot
I used Amazon EC2 to run the script and send me the CSV via email. I had to crawl through around 200GB of data and thus used one of the large instances to parallelize the parsing. I had to execute numerous parsing attempts to get the data right, the overall processing duration was 45 minutes. I don't know what I could have done without Amazon EC2. It was worth every buck I paid for it.
Splunk is a product for this sort of thing.
I have not used it my self though.
http://www.splunk.com/
The open source data mining and web mining software RapidMiner can import both Apache web server log files as well as CSV files and it can also import and export Excel sheets. Rapid-I offers a lot of training courses for RapidMiner, some also on web mining and web usage mining.
In the interest of full disclosure, I've not used any commercial tools for what your describing.
Have you looked at LogParser? It might be more manual than what your looking for, but it will allow you to query many different structured formats.
As for the graphical aspect of it, there is some basic charting capabilities built in, but your likely to get much more mileage piping the log parser output into a tabular/delimited format and loading into Excel. From there you can chart/graph just about anything.
As for cross joining different data sources, you can always pump all the data into the database where you'll have a richer language for querying the data.
What you're looking for is a "data mining framework", i.e. something which will happily eat gigabytes of somewhat random data and then lets you slice'n'dice it in yet unknown ways to find the gold nuggets buried deep inside of the static.
Some links:
CloudBase: "CloudBase is a high-performance data warehouse system built on top of Map-Reduce architecture. It enables business analysts using ANSI SQL to directly query large-scale log files arising in web site, telecommunications or IT operations."
RapidMiner: "RapidMiner aleady is a full data mining and business intelligence engine which also covers many related aspects ranging from ETL (Extract, Transform & Load) over Analysis to Reporting."