Scraper to Database visualiser connector - web-scraping

I'm using Dexi.io to scrape some data that outputs to Google Drive as a CSV, that gets parsed (through a Google sheets script) and added to a Native sheet (all automatically).
I'd like to push my data (automatically) to a "database Visualizer" of some sort (using knack.com currently) that allows me to display the data (in Table format) with some options to filter, sort and dig deeper; all protected by login creds that I manage.
I tried using Zapier to automate the Google Sheets to Knack integration, but Knack only has an option to "Create New Records" through Zapier and not "Update Records". (Updating records exists as an API endpoint)
I need help proceeding as I'm not a developer and am starting to hit the limits of my capabilities.
Could someone please recommend a tool (that integrates with Sheets, updates data periodically and lets me control the domain and login creds) or the optimal way to proceed with this? (I'd gladly hire a freelancer to help me build this out optimally)
Some more, potentially relevant, info: Dexi.io can output through FTP, Drive, Box or Amazon S3 (remember, not a dev :$)

kintone is a "database Visualizer" similar to Knack, and they have actions to update records.
https://zapier.com/zapbook/kintone/
There are two options to update records as there are two ways in which the unique key can be defined.
Each record in kintone has a "Record ID" associated with it - this is an autonumber made by kintone. You can specify this as the key to update, in which case you would use the "Update Record By Record ID" action.
If you would prefer though to set your own unique key and use that as the key to update, you can define that unique key in your database (I guess the data you are scraping has its own ID for each record). In this case, you can place a "Text (single-line)" field in your database, open up the options and select "Prohibit duplicate values" which will make this field into a unique field - meaning that no two records can hold the same value.
Once you set that field up (and update your kintone App settings), you can select this field to be the unique key to update for the "Update Record By Update Key" Action (the "Update Key" in the action name is referring to the unique key that you just made).
And yes, kintone gives you control over login creds, and each login cred can have different view/add/edit permissions over each record you have in your app.
You can also set a custom subdomain name, but the domain name will have to be kintone.com i.e. you can have a https:/ /mycustomname.kintone.com sort of name.
Hope this helps.

Related

InfoPath - How can I read a people picker field and query AD to load additional fields related to the person in the field.

How can I read a people picker field and query AD to load additional fields related to the person in the field. Example: Employee Name; load information want to load email address, phone number. the InfoPath form is being used with Nintex Workflow and SharePoint 2010.
I have searched and have not been able to find answer.
Thanks
D
Unfortunately I don't have enough reputation points yet to leave a comment, but your question really doesn't provide enough detail. So any answer provided is going to be based on assumptions. You don't even clarify what version of InfoPath you're using.
That said, a good place to start is to create a data connection to receive data. You'll need to select the web service option and will then need to enter in the web server address. The address will probably be in the following format:
http://yourservernamehere/_vti_bin/userprofileservice.asmx?wsdl
Replace yourservernamehere with the address of your SharePoint server. Then, you'll need to select GetUserProfileByName as the operation you need. Just keep on clicking next and then finish to complete the connection.
You will then have to view the data source within InfoPath to see what fields are available and map the ones you want to the fields you want prepopulated on your form.
All this is based on my own assumptions, so I can't guarantee it will work in your scenario. Happy to assist if you still need help and are able to provide more details.
To autocomplete you can use your e-mail or phone number fields, with a new action rule. This is due to people picker fields not allowing any action rules applied to them.
The web service option mentioned in another answer unfortunately no longer works in SharePoint Online. Please use a data connection the hidden User Information List located on the stem of your SharePoint site instead.
Condition:
Use the condition DisplayName is not blank by using "Select a field or group..." in advanced view and selecting your people pickers DisplayName field
Actions:
Set a field's value
Field: User Information List data connection queryFields DisplayName of people picker
Value: your forms DisplayName of people picker
Query using a data connection: User Information List data connection
Set a field's value
Field: your forms email or phone number field
Value: data fields > Work_email / Work_phone of your data connection

ASP.NET How to Apply Roles & Members Read/Write Securtiy to Pages, Sections, Fields, & Records

I've built a number sites using classic ASP type security and have continued using the same methods in ASP.NET. However, I now am looking for recommendations for a better or best practice way of doing it. I've read a few resource online but have not seen anything comprehensive enough that's applicable to what I'm trying to do. What I'm trying to do is apply user specific security that determines that user's access to specific pages, sections on that page, and fields in each section. It also needs to restrict access to various records as well and determine whether it's read or write privileges.
For those interested, here's how I've done it so far:
Because I lacked the know-how, here's how I did it using the old ASP classic way...
First, on the database side I have 4 relevant tables: tblUsers, tblRoles, tblPages, tblRecords (haven't gotten to sections and fields yet). Each user can belong to a "role" and the role then determines what pages they can access as well as what records they can access in various tables (there are a number of other tables in the db as well e.g. tblCustomers, tblSales, etc...). Each table includes these fields:
tblUsers: UserID, UserName, UserPwd
tblRoles: RoleID, RoleName, UserID
tblPages: PageID, PageName, RoleID
tblRecords: RecordID, RecordTable, RoleID
Now on the application side here's what I've done:
First, my login page is based on 1) looking up the user name and password in the tblUsers table and 2) if found, setting a session variable named "UserLoggedIn" = true. Then on every page load event I check if the UserLoggedIn session is set to true, if so, continue... if not clear all session variables (log out) and send the user back to the login page. The old classic ASP way of doing it.
Second, with the login set up, to control page access, when the user is logged in I created another session variable that holds a pipe delimited string of all the pages that user can access (e.g. Session("PageAccess") = "{1|3|10|8}"). Then in each page's load event I've explicitly added a variable/constant that uniquely identifies that page (e.g. pageone.aspx has Dim PageID As String = 1). Then I check to see if the PageID matches an ID stored in the PageAccess session. If it does, continue... If it doesn't I send them to the home page.
Third/Last, for the records access, I did the same thing. When the user is logged in I created a session variable that hold a pipe delimited string of all the records the user could access along with the table it's related to (e.g. Session("RecordAccess") = "{tblCustomrs||1|5|7}" and checked it and applied it basically the same way as the page session variable.
My Solution is :(it worked for my projects)
tables : tblUser , tblRole ,
tblUserInRole : userid,roleid,username,password (this design help you can assign more than one role to a user)
tblrole, tbrules(your Access Controls for example pages)
tblRulesInRole : roleid , ruleid,Editable,Viewable,printable,Deletable
For Implement Access Control in every request and response ,you should Create HttpModule......

How to grant a user access for specific time?

I am building a website users have to pay for different functions. For example the user have payed for "packet 1" for the current month and it includes "short messages". How can I check, if a user is allowed to send sms? Updating the user/role relations every night or is there anything more dynamic?
You could add custom voter, where you can check if a user is allowed to use certain service. See example here:
http://symfony.com/doc/current/cookbook/security/voters.html
You can create a table in a database which has a foreign key to a table of which contains different packages, a foreign key to your user table and finally a column which indicates the expire date of the package.
If the package is not found for that user and for that expire date then the user is not allowed to send sms.
You can use some other storage procedures but using a database makes most sense.
Basically: You don't want to update anything, you simply want to store certain information like the expire date (and maybe the starting date if you want to change the expire duration) and compare to that selected value.

Best method to retrieve Active Directory list using ASP.NET

I am fairly new to ASP.NET programming. I am designing a web project which will maintain employees information, such as the approval schema, staff inventories, claims, etc. The database will record the employee ID as the key. Currently there is no local table storing the mapping of the employee ID and employee name. These information will be retrieved from the Active Directory.
The new system will allow user to do employee lookup e.g. based on name or ID and generate report e.g. list of employee claims of the month. The lookup can be achieved by directly accessing the AD but I don't think it's a good method for generating list of employees/reports. Hence, I'm planning to download the AD list to local database.
My questions are:
1. Is downloading AD list to local database the right method for this situation? Is there any other alternatives to achieve this?
2. Shall I go with downloading the AD list, how to update it on regular basis? I can only think of clearing the table and reimport the whole list again.
Any advises will be much welcomed.
take a look at http://linqtoad.codeplex.com/ I've used it before with great success
"The new system will allow user to do employee lookup e.g. based on name or ID and generate report e.g. list of employee claims of the month."
Sounds like you do not need all employees from AD at once. In that case I would not download the AD list at all, as that would put you in a situation where you have to deal with your database being out of date.
Just start with retrieving the data from AD on request and only think about optimizations if you encounter performance problems.

ASP.NET user browse selected/restricted raw data from sql-server

In an ASP.NET WebForms application I would like to allow the end-user to browse selected raw data in an sql-server database.
However, I would like to restrict access for the user to only view some of the data based on the username.
I'm not sure how to do this in a way that is possible for the user to understand, since SQL is not necessarily known to the user.
What options do I have here?
As a basis for this I have considered creating one sql function per table in question. That function should return the data that the user is allowed to view, e.g.,
CREATE FUNCTION ufn_RawData_Employee(#username nvarchar(256))
RETURNS TABLE
AS ( SELECT * FROM Employee
WHERE [#username is allowed to view the given Employee] )
In a webpage the end-user might then type an SQL-like statement like
SELECT Name, HireDate FROM ((Employee))
where (([TableName])) then could be replaced by ufn_RawData_[TableName]([UserName]) before calling the database.
(For security reasons such calls could then be performed by a sql user whose only permissions are SELECT permissions to these functions.)
However, this approach might be too difficult for the end-user. I would like to know if an easier/user-friendlier solution exists for the end-user to browse selected raw data?
If you are only showing the user data from one table, or one view (which would probably be more useful) then yes you could store the name of that view in a table and retrieve it with a function. You could then display the data in pages, and make sure your standard select scripts have a built in search function if necessary.
There is no need for the user to write SQL if they are only getting data from one table or view. If you need to provide multiple potential tables/ views, then let them choose from a drop down, but it doesn't sound worthwhile to allow them to write their own SQL queries.

Resources