I have a question regarding Electron, since I have very limited understanding of the framework.
So what am I trying to achieve:
I am developing some sort of extension for an existing electron app, that exposes no API for my use case. The plan is to react on certain events that occure in the electron app. Those events are retrieved and transmitted via network. So my idea is to analyze the network traffic and wait for the relevant payloads.
Nearly everything is transmitted via TLS, why I am starting the electron app with environment variable "SSLKEYLOGFILE". That causes the app to write it's tls secrets into a file which can be analyzed by Wireshark. This works to some extend. I am able to decrypt a good portion of the traffic.
But there are also streams that originate from the electron app that cannot be decrypted because the secrets are not written to the log file. These streams also ignore when I start the app with "--proxy-server". So I suspect there is some kind of isolated part in the application that does not apply the provided configuration supplied to electron wrapper.
Do you have any hints for me how I can further investigate on this topic?
Related
I'm a bit of noob when it comes to deploying web apps and wanted to make sure a little app I'm building will work with the tech I'm trying to use.
I have some experience with flask, but have only ever used the test server. My understanding is that with nginx or apache, if I write a flask app, each user who visits my website could get a different instance of the flask app, exactly how that will work is a little confusing to me.
The app I want to make is similar to chatrooms/a game like "among us". When a user comes to the website, they join a big "lobby" and can either join a "room" that already exists, or launch a new room and generate a code/ID that they can pass to their friends so that their friends can join the same session (I think a socketio "room" can be used for this).
However, if each client is connected to their own flask instance, will every server instance be able to see the "rooms" on the other instances? Suppose my app becomes really popular and I want to scale the lobby across multiple machines/AWS instances in the future, is there anything I can do now to ensure this works? Or is scaling across multiple machines equivalent to scaling across instances on a single machine as far as the flask-socketio/nginx stack is concerned.
Basically, how do I ensure that the lobby part of the code is scalable. Is there anything I need to do to ensure every user has the ability to connect to rooms with other users even if they get a different instance of the flask app?
I will answer this question specifically with regards to the Socket.IO service. Other features of your application or third-party services that you use may need their own support for horizontal scaling.
With Flask-SocketIO scaling from one to two or more instances requires an additional piece, a message queue, which typically is either Redis or RabbitMQ, although there are a few more options.
As you clearly stated in your question, when the whole server is in a single instance, data such as which room(s) each connected client is in are readily available in the memory of the single process hosting the application.
When you scale to two or more instances, your clients are going to be partitioned and randomly assigned to one of your servers. So you will likely end up having the participants that are in a room also spread across multiple servers.
To make things work, the server instances all connect to the message queue and use messages to coordinate complex actions such as broadcasts to a room.
So in short, to scale from one to more instances, all you need to do is deploy a message queue, and change the Flask-SocketIO server to indicate the location of the queue. For example, here is the single instance server instantiation:
from flask_socketio import SocketIO
socketio = SocketIO(app)
And here is the initialization with a Redis message queue running on localhost's default 6379 port:
from flask_socketio import SocketIO
socketio = SocketIO(app, message_queue='redis://')
The application code does not need to be changed, Flask-SocketIO does all the coordination between instances for you by posting message on the queue.
Note that it does not really matter if the instances are hosted in the same server or in different ones. All that matters is that they connect to the same message queue so that they can communicate.
As per the KAA references, I understand that once should only use the Notification feature, When it required to send data from server (External apps) to endpoints and Events are only used when there is a need for endpoint to endpoint communication (kind of device binding requirement)
So, To achieve request/response functionality using KAA. I need to implement any hybrid solutions like as below.
1) In my server, I can run one KAA SDK instance and use the event feature for request to the endpoint and response from the endpoint.
OR
2) From my server, I use the notification REST API for request and get the response back through the data logger feature using any in-build appender by configuring "LogUploadStrategy" as to uploads every log record as soon as it is created.
Notes For Point 1
As per Andrew, Solutions Architect of Kaa IoT platform
"You can always embed an SDK to a standalone application and host in
on the same server where kaa-node is present. This application may
receive REST API calls and forward them to particular endpoints via
Kaa events feature. However, this is useful for test purposes. I
would not recommend this solution in production because it is hard to
scale and has potential security issues"
Notes For Point 2
It satisfies the KAA reference document as well as Andrew's suggestion for request only but how can i achieve the response.
Questions For Point 1
1) What causes to scale the application and what type of security issues it faces even through it uses RSA 2048 encryption for communication?
2) Can we embed more then one SDK in standalone application and host in on the same server where kaa-node is present.
Questions For Point 2
3) if device sends the notification response along with the telemetry data, can it increase the latency and any other performance issue.
Common Questions
4) Which one is the better approach to achieve request/response functionality?
Any help or suggestion is really appreciated.
1) What causes to scale the application and what type of security issues it faces even though it uses RSA 2048 encryption for communication?
It makes the EP on the server side as a single point of failure and does not allow load balancing.
About security issues, Andrew meant: This application may receive REST API calls and this forces one to provide additional security for this REST API calls and better use your first hybrid solutions using solely event feature.
2) Can we embed more then one SDK in standalone application and host in on the same server where kaa-node is present.
No, you can't use more than one SDK in one application, but you can run a couple of instance on one machine in different directories
in order to prevent collisions of autogenerated security keys and other files.
3) if device sends the notification response along with the telemetry data, can it increase the latency and any other performance issue.
Of course, you will face some delays if start sending very frequently and big portions of data on both sides. If you have a lot of devices that sends in total a big amount of telemetry data, you can increase performance on the server side by start-up KAA in the cluster mode or add new nodes for processing requests.
4) Which one is the better approach to achieve request/response functionality?
The second hybrid solution – data collection and notification features. This doesn't cause any problem with scale and you can easily launch Kaa server in cluster mode.
I want to create a program for my desktop and an app for my android. Both of them will do the same, just on those different devices. They will be something like personal assistants, so I want to put a lot of data into them ( for example contacts, notes and a huge lot of other stuff). All of this data should be saved on a server (at least for the beginning I will use my own Ubuntu server at home).
For the android app I will obviously use java and the database on the server will be a MySQL database, because that's the database I have used for everything. The Windows program will most likely be written in of these languages: Java, C#c C++, as these are the languages I am able to use quite well.
Now to the problem/question: The server should have a good backend which will be communicating with the apps/programs and read/write data in the database, manage the users and all that stuff. But I am not sure how I should approach programming the backend and the "network communication" itself. I would really like to have some relatively easy way to send secured messages between server and clients, but I have no experience in that matter. I do have programming experience in general, but not with backend and network programming.
side notes:
I would like to "scale big". At first this system will only be used by me, but it may be opened to more people or even sold.
Also I would really like to a (partly) self programmed backend on the server, because I could very well use this for a lot of other stuff, like some automation features in my house, which will be implemented.
EDIT: I would like to be able to scale big. I don't need support for hundreds of people at the beginning ;)
You need to research Socket programming. They provide relatively easy, secured network communication. Essentially, you will create some sort of connection or socket listener on your server. The clients will create Sockets, initialize them to connect to a certain IP address and port number, and then connect. Once the server receives these connections, the server creates a Socket for that specific connection, and the two sockets can communicate back and forth.
If you want your server to be able to handle multiple clients, I suggest creating a new Thread every time the server receives a connection, and that Thread will be dedicated to that specific client connection. Having a multi-threaded server where each client has its own dedicated Thread is a good starting point for an efficient server.
Here are some good C# examples of Socket clients and servers: https://msdn.microsoft.com/en-us/library/w89fhyex(v=vs.110).aspx
As a side note, you can also write Android apps in C# with Xamarin. If you did your desktop program and Android app both in C#, you'd be able to write most of the code once and share it between the two apps easily.
I suggest you start learning socket programming by creating very simple client and server applications in order to grasp how they will be communicating in your larger project. Once you can grasp the communication procedures well enough, start designing your larger project.
But I am not sure how I should approach programming the backend and
the "network communication" itself.
Traditionally, a server for your case would be a web server exposing REST API (JSON). All clients need to do http requests and render/parse JSON. REST API is mapped to database calls and exposes some data model. If it was in Java, it would be Jetty web server, Jackson Json parser.
I would really like to have some relatively easy way to send secured
messages between server and clients,
Sending HTTP requests probably the easiest way to communicate with a service. Having it secured is a matter of enabling HTTPS on the server side and implementing some user access authentication and action authorization. Enabling HTTPS with Jetty for Java will require few lines of code. Authentication is usually done via OAuth2 technique, and authorization could be based on ACL. You may go beyond of this and enable encryption of data at rest and employ other practices.
I would like to "scale big". At first this system will only be used by
me, but it may be opened to more people or even sold.
I would like to be able to scale big. I don't need support for
hundreds of people at the beginning
I anticipate scalability can become the main challenge. Depending on how far you want to scale, you may need to go to distributed (Big Data) databases and distributed serving and messaging layers.
Also I would really like to a (partly) self programmed backend on the
server, because I could very well use this for a lot of other stuff,
like some automation features in my house, which will be implemented.
I am not sure what you mean self-programmed. Usually a backend encapsulates some application specific business logic.
It could be a piece of logic between your database and http transport layer.
In more complicated scenario your logic can be put into asynchronous service behind the backend, so the service can do it's job without blocking clients' requests.
And in the most (probably) complicated scenario your backend may do machine learning (for example, if you would like you software stack to learn your home-being habits and automate house accordingly to your expectations without actually coding this automation)
but I have no experience in that matter. I do have programming
experience in general, but not with backend and network programming.
If you can code, writing a backend is not very hard problem. There are a lot of resources. However, you would need time (or money) to learn and to do it, what may distract you from the development of your applications or you may enjoy it.
The alternative to in-house developed of a backend could be a Backend-as-a-Service (BaaS) in cloud or on premises. There are number of product in this market. BaaS will allow you to eliminate the development of the backend entirely (or close to this). At minimum it should do:
REST API to data storage with configurable data model,
security,
scalability,
custom business-logic
Disclaimer: I am a member of webintrinsics.io team, which is a Backend-as-a-Service. Check our website and contact if you need to, we will be able to work with you and help you either with BaaS or with guiding you towards some useful resources.
Good luck with your work!
When using java APNs, if using the same program to send messages to different applications, it is required to use different certificates. Does switching certificates causes disconnection/reconnection and is it considered a bad practice? (as APNs would like to have persistent connections thus preventing denial of service).
I don't think it's possible to switch a certificate for an existing connection. At least I'm quite sure it's not possible in Java SSLSocket (since it is constructed by an SSLSocketFactory, and you use the certificate in one of the parameters required for constructing that factory).
You should maintain an open socket for each application you are sending notifications to. This way you don't have to close the socket and open a new one each time you need to switch certificates.
is there a way for a server to push some data to a client, wirelessly and seamlessly, which may be Windows(Phone), iPhone, Mac, or Android device, without any OS integration?
If so, what's the best design pattern to do this, and what are the best technologies to go about this?
Push technology is simply a methodology of the server initiating the transfer of data, rather than the client asking the server for it.
Apple makes push technology relatively easy to use by providing such functionality built-in on the OS. As well as Android through the Google Cloud Messaging for Android. Windows, however, does not.
Apple push notifications and Google's messaging for Android is seemingly magical and/or functionality that the OS needs to handle; however, this isn't necessarily the case. The advantage of having it "integrated" in the OS, is the same as having a framework handle the functionality for you.
Speaking in technical terms, push technology is a long-lived connection from the client to the server that accepts messages. These messages would be considered pushed messages, since the client did not make an individual request for them.
The main thing to keep in mind when implementing push technology yourself, is that the client is in charge of keeping that long-lived connection alive as much as possible. Because client IP addresses can change between disconnects, servers are not guaranteed that a client's address will be persistent across disconnects. Moreover, clients can be connected from behind a firewall, making it impossible for a server to reach the client.
For comparison, pull technology is the more traditional process of a client connecting to a server and requesting data.
Your best bet for Apple iOS will be using their push notification service.
For Android devices you should use the Google Cloud Messaging for Android. Alternatively, you can create your own background service to handle the messaging; here's a guide.
For Windows (desktop at least), you will have to create your own service to perform such duty. Here's an MSDN guide explaining how to create a Windows Service using Visual Studio (VB and C#). There might be frameworks already built that handle such messaging on Windows, however, I don't know of any.
Use WebSocket (with or without socket.io).
In the future, you could use WebRTC.
With Websockets, the setup is really simple. The client (a user agent, like a browser or a WebView) connects to the Websocket server, over http(s) (less problems with firewalls) and that's it. There's a bidirectional socket with an event-based API.
If by "OS integration" you mean "write special code for each platform" then the answer is no.
As you mentioned, you would like file system access, and background processing. That combination is not available in a cross-platform way at this moment.
If by "OS integration" you meant "without having to wait for apple/google/ms to provide the ability" then the answer is yes/maybe.
All the popular platforms have Push notifications and background processing support, as long as you code it the way each particular platform expects it.
But file system access will be limited to what restrictions the platform places on you. For instance in ios and win8(phone) there is no wy to write or read a file outside of your own apps private file structure. For security reasons, you cannot access the file system of other apps.
UPDATE:
The general pattern here is to release an app for every platform you want to support.
The app will register itself with its respective platform's push notification service.
You will write generic server side code to accept the data you want to push to all your client devices. Then you will invoke the respective push API's for each platform you support, causing the client devices to wake up and trigger the app that you provided to respond.
When the app opens, you get the app to contact your server and download the full data "the push notification being just the wakeup call for your app"
This way you can easily tell how which of your devices have received the data.
Each platform specific app must save the data to its own local storage and provide a way for the data to be shared via the methods supported by its respective platform.
On IOS it can be as simple as supporting the "Open In.." paradigm.
On W8Phone, you'll have to publish the data via one of the available "sharing contracts"
And so forth for every platform you want to support.
This is the general pattern at the moment. There are some caveats. On IOS, the app will nt automatically start when a push notification is received. This means your app will only download the whole of the data when the app is opened by the user.
A mobile app also cannot generally run indefinitely in the background. This means that once the app is started, you have a limited window for push notifications to be automatically processed by your app. After the allowed "background time". The app will close and any push notifications beep on the device, but wont open the app until the user taps on one of the notifications or opens the app directly.
Technically you could use XMPP Libraries, it is meant for implementing chat system (msn, gtalk, facebook chat, etc...) but this could work well as a push message system because it is opensource and well built to handle all the cases you never thought of. Also you could host your own server and send push message that way...
Why not use a webservice? In my previous project I used webservice to deliver data from mysql database. The webservice I used was nusoap. On the client side I used kSoap library for Android. Hope this helps.
if you want to receive and send real time communication between a server and client (irrespective of the device or OS), i would highly recommend you use XMPP technology because it is designed for the sort of things you're asking for.
The Extensible Messaging and Presence Protocol (XMPP) is an application profile of the Extensible Markup Language [XML] that enables the near-real-time exchange of structured yet extensible data between any two or more network entities. The core features of XMPP defined in [XMPP‑CORE] provide the building blocks for many types of near-real-time applications, which can be layered on top of the core by sending application-specific data qualified by particular XML namespaces.
http://xmpp.org/rfcs/rfc6121.html#intro - that is the latest RFC which will give you a good starting point.