CSV to user custom report (Dygraph) - filereader

I'm trying to create a good looking report from data which I get from Codesys PLC.
I'm going to create HTML page where you can put a file with data, and then it will be parsed into specific fields.
Codesys PLC writes CSV file of the following format:
under section some general data of the process
under trends which I going to visualize with Dygraph.
I tested Dygraph when CSV file consists only with GRAPH data (starts directly from captions row).
I created new dygraph and give it the CSV as File by FileReader.
Now I'm struggling how to hanle the section.
I can read it line by line, but then I need to hanle section as File again to put it into Dygraph.
So the main question is how to read a File as CSV file from specific position?
I tested some code, which I collects form some examples.
I'm sorry for that, I know it is not good but I'm a new with jscript and web programming.
But it works for Graph showing.
CSV File:
REPORT
StartDate, 2020/02/05 12:26:53
EndDateDate, 2020/02/06 12:26:53
Status, Finished
AverageTemperature, 123.45
AveragePressure, 0.0567
GRAPH
Time,Temperature,Pressure
2020/02/05 12:26:53,54.53312,0.5453312
2020/02/05 12:26:55,40.7445,0.1613245
2020/02/05 12:26:56,7.167816,0.03520107
2020/02/05 12:26:57,78.76286,0.03732681
2020/02/05 12:26:58,22.20001,0.06941796
2020/02/05 12:27:00,34.79605,0.7027158
<div id="graph"></div>
<input type="file" id="files" name="files[]" multiple />
<output id="list"></output>
<table id="outputTable">
</table>
<script>
var table = document.getElementById('outputTable');
function parseCSV(text, lineTerminator, cellTerminator) {
g = new Dygraph(
document.getElementById("graph"),
text,
{
series: {
'temperature': {
axis: 'y'
},
'pressure': {
axis: 'y2'
},
},
axes: {
y: {
labelsKMB: true,
independentTicks: true
},
y2: {
// set axis-related properties here
labelsKMB: true,
independentTicks: true
}
},
ylabel: 'Primary y-axis',
y2label: 'Secondary y-axis',
});
//break the lines apart
var lines = text.split(lineTerminator);
for(var j = 0; j<lines.length; j++){
if(lines[j] != ""){
//create a table row
var tableRow = table.appendChild(document.createElement('tr'));
//split the rows at the cellTerminator character
var information = lines[j].split(cellTerminator);
for(var k = 0; k < information.length; k++){
//append the cell to the row
var cell = tableRow.appendChild(document.createElement('td'));
cell.appendChild(document.createTextNode(information[k]));
}
}
}
}
function handleFileSelect(evt) {
var files = evt.target.files; // FileList object
// Loop through the FileList and populate the 'outputTable' with the data
for (var i = 0, f; f = files[i]; i++) {
var reader = new FileReader();
// Closure to capture the file information.
reader.onload = (function(theFile) {
return function(e) {
//call the parse function with the proper line terminator and cell terminator
parseCSV(e.target.result, '\n', ';');
};
})(f);
// Read the file as text
reader.readAsText(f);
}
}
document.getElementById('files').addEventListener('change', handleFileSelect, false);
</script>

Related

Extracting data from a web page to Excel sheet

How can I extract information from a web page into an Excel sheet?
The website is https://www.proudlysa.co.za/members.php and I would like to extract all the companies listed there and all their respective information.
The process you're referring to is called web scraping, and there are several VBA tutorials out there for you to try.
Alternatively, you can always try
(source: netdna-ssl.com)
I tried creating something to grab for all pages. But ran of time and had bugs. This should help you a little. You will have to do this on all 112 pages.
Using chrome go to the page
type javascript: in the url then paste the code below. it should extra what you need. then you will have to just copy and paste it in to excel.
var list = $(document).find(".pricing-list");
var csv ="";
for (i = 0; list.length > i;i++) {
var dataTags = list[i].getElementsByTagName('li');
var dataArr = [];
for (j = 0; dataTags.length > j;j++) {
dataArr.push(dataTags[j].innerText.trim());
}
csv += dataArr.join(', ') + "<br>";
}
you will get something like this
EDITTED
use this instead will automatically download each page as csv then you can just combine them after somehow.
Make sure to type javascript: in url before pasting and pressing enter
Also works with chrome, not sure about other browsers. i dont use them much
var list = $(document).find(".pricing-list");
var csv ="data:text/csv;charset=utf-8,";
for (i = 0; list.length > i;i++) {
var dataTags = list[i].getElementsByTagName('li');
var dataArr = [];
for (j = 0; dataTags.length > j;j++) {
dataArr.push(dataTags[j].innerText.trim());
}
csv += dataArr.join(', ') + "\n";
}
var a = document.createElement("a");
a.href = ""+ encodeURI(csv);
a.download = "data.csv";
a.click();

Google Sheets: delete rows containing specified data

I'm new to Java scripting and Google Apps Scripts so i am sorry if this has already been answered. I was not able to find what i was looking for over the last few months of working on this project.
I am working on a variant of the scripts here:
Delete row in Google Sheets if certain "word" is found in cell
AND
Google Sheet Script - Find Value in Col and Delete Row
I want to create a button, or menu, that will allow someone to enter specific data, and have each row in the spreadsheet containing that data deleted.
I have a test sheet here that illustrates the data i'm working with, formulas i'm using, and has the beginning of the script attached to it:
https://docs.google.com/spreadsheets/d/1e2ILQYf8MJD3mrmUeFQyET6lOLYEb-4coDTd52QBWtU/edit?usp=sharing
The first 4 sheets are pulling data from the "Form Responses 1" sheet via a formula in cell A:3 in each sheet so the data would only need to be deleted from the "Form Responses 1" sheet to clear it from the rest of the sheets.
I tried working this in but i do not think i am on the right track.
https://developers.google.com/apps-script/guides/dialogs
I also posted this on Google Docs Help Forum 60 days ago, but have not received any responses.
Any help would be greatly appreciated.
There's a few steps. For usability of UI this takes a little longer code. In concise form:
The user activates a dialog and enters a string.
Rows w/ the string are deleted (with error handling and confirmation)
(Hopefully this gets you started and you can tailor it to your needs)
Function that initiates the menu:
function onOpen(){
SpreadsheetApp.getUi()
.createMenu('My Menu')
.addItem('Delete Data', 'deleteFunction')
.addToUi();
}
The main workhorse:
function deleteFunction(){
//declarations
var sheetName = "Form Responses 1";
var ss = SpreadsheetApp.getActive();
var sheet = ss.getSheetByName(sheetName);
var dataRange = sheet.getDataRange();
var numRows = dataRange.getNumRows();
var values = dataRange.getValues();
var delete_string = getUIstring();//open initial UI, save value
if (delete_string.length < 3) return shortStringError()//UI to protect your document from an accidental entry of a very short string.
//removing the rows (start with i=2, so don't delete header row.)
var rowsDeleted = 0;
for (var i = 2; i <= numRows; i++){
var rowValues = values[i-1].toString();//your sheet has various data types, script can be improved here to allow deleting dates, ect.
if (rowValues.indexOf(delete_string) > -1){
sheet.deleteRow(i - rowsDeleted);//keeps loop and sheet in sync
rowsDeleted++;
}
}
postUIconfirm(rowsDeleted);//Open confirmation UI
}
Isolated UI functions to help make above function more concise:
function getUIstring(){
var ui = SpreadsheetApp.getUi();
var response = ui.prompt("Enter the target data element for deletion")
return response.getResponseText()
}
function postUIconfirm(rowsDeleted){
var ui = SpreadsheetApp.getUi();
ui.alert("Operation complete. There were "+rowsDeleted+" rows deleted.")
}
function shortStringError(){
var ui = SpreadsheetApp.getUi();
ui.alert("The string is too short. Enter a longer string to prevent unexpected deletion")
}
I'll just show a way to delete the cell value if it matches your search criteria. It's up to you to connect it to buttons ,etc.
You'll loop through a Sheet Range. When you find the word match, delete it using clearContent()
function deleteSpecificData() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getSheets()[0];
var range = sheet.getRange("Sheet1!A1:C4");
var values = range.getValues();
var numArray = [1,2,3,4,5,6,7,8,9];
var deleteItem = "Garen";
Logger.log(range);
for(var i=0; i< values.length; i++){
for(var j=0; j<values[i].length; j++){
if(values[i][j] == deleteItem){
var row = numArray[i];
var col = numArray[j];
var range = sheet.getRange(row,col).clearContent();
}
}
}
}
Before:
After:

JqxWidgets: Export nested grid

While working with JqxWidges I met a problem with exporting nested grids which use one JSON as a source file. The common solution doesn't work. Actually it exports only parent grid colums.
$("#excelExport").click(function () {
$("#jqxGrid").jqxGrid('exportdata', 'csv', chartName + ' ' + date);
});
One of the existing solutions (http://www.jqwidgets.com/community/reply/reply-to-export-data-from-a-nested-grid-13/) propose to push nested rows into data array while calling initrowdetails function.
Yes it works! But only for nested grids and in case when this grid was selected.
So, from this step I am moving to next aproach:
To collect all necessary data into array using initial JSON (prevent you from gathering only separate selected data);
To initialise parent grid columns with all existing data and mark nested columns as hidden. Then when export don't forget to add true parameter to export both non/hidden columns;
Use standard export with custom array parameter;
That's it!
Data collecting:
var toExport = data.allClientsCountChart;
var exp = new Array();
for(var i in toExport){
var client = {};
var countr = toExport[i].countries;
client[labels.clientType]=toExport[i].clientType;
client[labels.clientTypeCount]=toExport[i].clientTypeCount;
exp.push(client);
for(var j in countr) {
var country = {}
var detailes = countr[j].clientDetails;
country[labels.countryType]=countr[j].countryType;
country[labels.clientsNumber]=countr[j].clientsNumber;
exp.push(country);
for(var d in detailes) {
var det = {}
det[labels.scriptName]=detailes[d].scriptName;
det[labels.clientsCount]=detailes[d].clientsCount;
exp.push(det);
}
}
}
Export:
$("#excelExport").click(function () {
$("#jqxGrid").jqxGrid('exportdata', 'csv', chartName + ' ' + date, true, exp, true);
}
And don't forget to set the fifth pafameter into true to export hidden columns.
No doubds, it looks hardcoded. But it works for me.
So, if you have a good solution - please leave a comment!!!

OpenLayer Popups for markers imported from google spreadsheet

I'm looking for a way to use framecloud type popup with my current setup. Unfortunately all my attempts have either not worked or will only work on the most recently placed maker.
In the course of trying to get it to work I have converted my original script from using Markers to using Vectors to placing the marker points (as I've seen that it's easier to customize vectors than markers.)
Now which ever one I can get to work I'll use, but after working on this for a few days I'm at my wits end and need a helping hand in the right direction.
My points are pulled from a google spreadsheet using tabletop.js. The feature is working how I wish it to, with the markers being placed on their respective layer based on a field I called 'type'.
While I have a feeling that might have been the source of my problem with the Markers type layer, I'm not sure how to fix it.
You can view the coding through these pages
(Links removed due to location change.)
Thanks for all help in advance.
I finally got it to work. For anyone in a similar situation here's my final code for the layers. I did change the names of the layers from what they are originally and blacked out the spreadsheet I used, but the changes should be noticeable.
//
//// Set 'Markers'
//
var iconMarker = {externalGraphic: 'http://www.openlayers.org/dev/img/marker.png', graphicHeight: 21, graphicWidth: 16};
var iconGeo = {externalGraphic: './images/fortress.jpg', graphicHeight: 25, graphicWidth: 25};
var iconAero = {externalGraphic: './images/aeropolae.jpg', graphicHeight: 25, graphicWidth: 25}; // Image is the creation of DriveByArtist: http://drivebyartist.deviantart.com/
var vector1 = new OpenLayers.Layer.Vector("1");
var vector2 = new OpenLayers.Layer.Vector("2");
var vector3 = new OpenLayers.Layer.Vector("3");
// Pulls map info from Spreadsheet
//*
Tabletop.init({
key: 'http://xxxxxxxxxx', //Spreadsheet URL goes here
callback: function(data, tabletop) {
var i,
dataLength = data.length;
for (i=0; i<dataLength; i++) { //following are variables from the spreadsheet
locName = data[i].name;
locLon = data[i].long;
locLat = data[i].lat;
locInfo = data[i].info;
locType = data[i].type; // Contains the following string in the cell, which provides a pre-determined output based on provided information in the spreadsheet: =ARRAYFORMULA("<h2>"&B2:B&"</h2><b>"&G2:G&"</b><br /> "&C2:C&", "&D2:D&"<br />"&E2:E&if(ISTEXT(F2:F),"<br /><a target='_blank' href='"&F2:F&"'>Read More...</a>",""))
locLonLat= new OpenLayers.Geometry.Point(locLon, locLat);
switch(locType)
{
case "Geopolae":
feature = new OpenLayers.Feature.Vector(
locLonLat,
{description:locInfo},
iconGeo);
vector1.addFeatures(feature);
break;
case "POI":
feature = new OpenLayers.Feature.Vector(
locLonLat,
{description:locInfo},
iconMarker);
vector2.addFeatures(feature);
break;
case "Aeropolae":
feature = new OpenLayers.Feature.Vector(
locLonLat,
{description:locInfo},
iconAero);
vector3.addFeatures(feature);
break;
}
}
},
simpleSheet: true
});
map.addLayers([vector1, vector2, vector3]);
map.addControl(new OpenLayers.Control.LayerSwitcher());
//Add a selector control to the vectorLayer with popup functions
var controls = {
selector: new OpenLayers.Control.SelectFeature(Array(vector1, vector2, vector3), { onSelect: createPopup, onUnselect: destroyPopup })
};
function createPopup(feature) {
feature.popup = new OpenLayers.Popup.FramedCloud("pop",
feature.geometry.getBounds().getCenterLonLat(),
null,
'<div class="markerContent">'+feature.attributes.description+'</div>',
null,
true,
function() { controls['selector'].unselectAll(); }
);
feature.popup.autoSize = true;
feature.popup.minSize = new OpenLayers.Size(400,100);
feature.popup.maxSize = new OpenLayers.Size(400,800);
feature.popup.fixedRelativePosition = true;
feature.popup.overflow ="auto";
//feature.popup.closeOnMove = true;
map.addPopup(feature.popup);
}
function destroyPopup(feature) {
feature.popup.destroy();
feature.popup = null;
}
map.addControl(controls['selector']);
controls['selector'].activate();
}

Google Maps v3 Shapes

I am trying to take a string which has shape option information and create the shape on my Google Map application.
The string is made by splitting an array that was built from a local text document.
The string appears as:
Circle{center: new google.maps.LatLng(38.041872419557094, -87.6046371459961),radius:5197.017394363823,fillColor: '#000000',strokeWeight: 1,strokeColor: '#000000',map:map};
The function I have to take such string and make the shape appears as:
function loadDrawings(evt)
{
var f = evt.target.files[0];
if (!f)
{
alert("Failed to load file");
}
else if (!f.type.match('text.*'))
{
alert(f.name + " is not a valid text file.");
}
else
{
var r = new FileReader();
r.onload = function (e)
{
var contents = e.target.result;
var drawings = [];
var drawing;
var drawingType;
var shape;
var shapeOptions;
drawings = contents.split(";");
for (i = 0; i < drawings.length - 1; i++) {
drawing = drawings[i].toString();
drawingType = drawing.substr(0, drawing.indexOf('{'));
if (drawingType == "Circle")
{
shapeOptions = drawing.substr(6); //UNIQUE TO CIRCLE
shape = new google.maps.Circle(shapeOptions);
shape.setMap(map);
}
};
}
r.readAsText(f);
}
}
My issue is shapeOptions as a string does not work in the above syntax for creating the Circle. However, if I take the contents of the string, which is:
{center: new google.maps.LatLng(38.041872419557094, -87.6046371459961),radius:5197.017394363823,fillColor: '#000000',strokeWeight: 1,strokeColor: '#000000',map:map}
And directly enter it, the shape appears.
Do I need a certain variable type for my shapeOptions for this to work? I know that the new google.maps. requires (), but I have had no luck creating a variable from my string. Am I missing something here?
Much appreciation for any help!
Your shapeOptions string is a JavaScript object literal, so you can eval() it to get the object:
shapeOptions = eval( '(' + drawing.substr(6) + ')' );
Since it has map:map in it, you don't need the subsequent setMap() call.
Also, you're missing a var for the i variable. I don't really recommend the coding style where all the var statements go at the top of a function. I find it error-prone; it's too easy to omit a var without noticing it. (I know some famous JavaScript experts insist that var at the top is the only way to do it, but they fail to see the tradeoffs involved.)
You don't need the .toString() on drawings[i]. It's already a string.
You have two different brace styles. Best to pick one and stick with it. For JavaScript, putting the { on a line by itself is not recommended, because this code will not do what you expect:
return // hoping to return an object literal - but it doesn't!
{
a: 'b',
c: 'd'
}
Whereas this code does work correctly:
return {
a: 'b',
c: 'd'
}
Since you are using FileReader, I think it's safe to assume you also have .forEach() available.
You can replace the code that uses .indexOf() and the hard coded length with a regular expression.
Putting all that together, you might end up with code like this:
var r = new FileReader();
r.onload = function( e ) {
e.target.result.split(";").forEach( function( drawing ) {
var match = drawing.match( /^(\w+)({.*})$/ );
if( ! match ) return; // unrecognized
var type = match[0], options = eval( match[1] );
switch( type ) {
case "Circle":
new google.maps.Circle( options );
break;
}
});
}
r.readAsText( f );
But you may be able to take it a step further. So far we're looking at a Circle (line breaks added for readability):
Circle{
center: new google.maps.LatLng(
38.041872419557094,
-87.6046371459961
),
radius:5197.017394363823,
fillColor: '#000000',
strokeWeight: 1,
strokeColor: '#000000',
map:map
}
With only a simple change, that could be executed as JavaScript directly. You just need the 'new google.maps.' at the beginning and () around the object literal:
new google.maps.Circle({
center: new google.maps.LatLng(
38.041872419557094,
-87.6046371459961
),
radius:5197.017394363823,
fillColor: '#000000',
strokeWeight: 1,
strokeColor: '#000000',
map:map
})
I assume you will have other drawing types as well? Will they all map directly to google.maps.* objects like Circle does? If so, you could simply do:
var r = new FileReader();
r.onload = function( e ) {
e.target.result.split(";").forEach( function( drawing ) {
eval( drawing.replace(
/^(\w+)({.*})$/,
'new google.maps.$1(\$2)'
) );
});
}
r.readAsText( f );

Resources