I want to use Isabelle on weaker laptops and delegate the heavy theorem search/proving to a server on the network. I would guess that this has been done before but I could not find tutorials or reports for this task.
The Isabelle System Manual describes how to run the Isabelle backend on its own. However I don't understand from the manual how to connect one of the existing frontends (e.g. Isabelle/jEdit) to such a process. The setup should ideally work for multiple users (and with theory files located on the user system).
The best I could achieve up to now was to run all of Isabelle/jEdit on the server and access it from a Linux laptop via SSH/X11-forwarding. This is cool but not exactly what I had in mind. Are there other approaches?
Related
I am exploring the possible solutions for orchestrating my flows across multiple services via some infrastructure. Searching shows me a few options such as Conductor, Camunda, Airflow etc.
I am wondering what would fit my use case better
One of my service is in Java, the other is in Python
I need to pass info to the Java service, then take the output and pass it to the Python service
Final output is then published to another queue
It feels like Conductor is a good choice, but would love to hear your inputs!
All options can fulfill the requirement stated. Think about further / future requirements. Is it only a data pipe? Is it about orchestrating a larger end-to end business process? Do you need support for long-running processes? Is end-to-end transparency in a graphical form a benefit? Is graphical process modelling in BPMN2 standard going to be a benefit? Are there going to be audit or reporting requirements? Or is it going to be a simple, isolated, technical solution?
This article gives a great overview of tools in the market and what their primary use cases are: https://blog.bernd-ruecker.com/understanding-the-process-automation-landscape-9406fe019d93
All listed tools might technically be able to execute your workflow (I have no experience working with Conductor & Camunda). A few characteristics on which a decision is usually made are:
open vs closed source
how do you define workflows? (e.g. Python code in Airflow. Others use e.g. JSON/XML/something custom)
does it come with a UI?
can it scale out in case my workloads start growing?
is it agnostic to any technology or limited to running certain technologies? (e.g. Oozie is built for scheduling jobs on Hadoop)
other requirements could be e.g. security, logging, monitoring, etc.
There are many orchestration-tool-comparisons on the internet, e.g. 1 or 2.
Introduction to Container Orchestration
The practice of automating the administration of container-based microservice applications across different clusters is known as container orchestration. Within corporations, this notion is gaining popularity. In addition, a variety of Container Orchestration technologies have become indispensable in the deployment of microservice-based applications.
Software development in the modern era is no longer monolithic. Instead, it generates component-based apps that run across many containers. These adaptable and scalable containers work together to accomplish a specified purpose or microservice.
Depending on the complexity of the application and other requirements like load balancing, they may span many clusters.
Containers encapsulate application code as well as its dependencies. To function efficiently, they receive the resources they require from physical or virtual hosts. When complicated systems are built as containers, clustering them for deployment requires adequate management and priority.
How to Choose a Container Orchestration Tool?
We've looked at a number of Orchestration Tools that you may examine when selecting which is ideal for your business. To do so, make sure to understand your company's requirements and operations. Then you'll be able to more readily weigh the benefits and drawbacks of each option.
Kubernetes
Kubernetes has a lot of features and is ideally suited for container and cluster management at the corporate level. Kubernetes is managed by a number of platforms, including Google, AWS, Azure, Pivotal, and Docker. As the containerized workload grows, you have a lot of options.
The biggest disadvantage is that it does not work with Docker Swarm and Compose CLI manifests. It might also be difficult to understand and set up. Despite these flaws, it is one of the most used systems for cluster deployment and management.
Docker Swarm
For individuals who are already familiar with Docker Compose, Docker Swarm is a better option. It's easy to use and doesn't require any additional software. Unlike Kubernetes and Amazon ECS, however, Docker Swarm lacks sophisticated features such as built-in logging and monitoring. As a result, it is better suited to small-scale businesses that are just starting started with containers.
Amazon ECS
If you're already familiar with Amazon Web Services, Amazon ECS is a great way to install and configure clusters. It's a quick and easy method to get started, and it scales to match demand. It also connects with a number of other AWS services. It's also excellent for small teams with limited resources for container maintenance.
One of its disadvantages is that it is incompatible with nonstandard deployments. It also contains ECS-specific configuration files, which complicates debugging.
I am a newbie with Xen.I want to know how does Xen work.
It's really a puzzle when facing the code and I don't know where to start.
Are there some easy articles for me?
Since you mention looking at the code, I assume you want to understand the technical details of Xen and not just merely how to start a VM.
As with all problems, start with something simple and then work your way up. Some pointers:
Be sure to have the prerequisite experience under your belt. In particular, strong C and Linux affinity, but also x86 paging and virtualized memory workings.
Make sure you have a sound grasp of the general Xen architecture. For instance, paravirtualized versus hardware-supported virtualization, the special role of the management domain (Dom0) compared to unprivileged domains (DomU), etc.
Investigate the the Xen components running in Dom0:
The Xen control library (libxc) which implements much of the logic relating to hypercalls and adds sugar around these (look in tools/libxc).
The swiss army knife for administrating Xen, namely the Xen light library (libxl). This library replaces the deprecated xm tool with the xl tool and takes care of all your maintenance tasks such as starting/stopping a VM, listing all running VMs, etc. For all these operations, it works in tandem with the aforementioned libxc. (Libxl lives in tools/libxl.)
The Xenstore is a tree-like data structure from which all running domains can retrieve and store data. This is necessary since all I/O goes through Dom0 (not the hypervisor!), and domains need to communicate with Dom0 how they are going to pass I/O along. (Look in tools/xenstore.) You can inspect the Xenstore with a tool such as xenstore-ls.
the blkback/netback kernel drivers which pass the data over shared channels to the VMs. (You will find these drivers in a recent Linux kernel (e.g. >= v3.0) that has so-called PVOPS support).
Take a look at the console daemon (tools/console). Note that sometimes the Qemu console is actually used. Qemu also comes in the pictures as a default backend for if you choose a file-backed virtual storage for a VM.
Experiment with the 'Xen-way' of inter-VM communication: Grant tables, event channels and the Xenstore. With these fundamentals you can create your own shared channel between VMs. You can do this, for example, with writing a kernel module that you use in two domains to let them talk to each other.
I can also give some pointers in the source that you can check out:
xen/xen/include/public/xen.h will give you a list of all the hypercalls with comments what they do.
xen/xen/include/xen/mm.h gives you an introduction to the different memory terminology used by Xen (i.e., real versus virtualized addresses and page numbers). If you don't grasp these differences, then reading the hypervisor code will surely be frustrating.
xen/xen/include/asm-x86/config.h gives an overview of the memory layout of Xen.
xen/tools/libxc/xenctrl.h exports a large list of interesting domain control operations, which gives an abstract view of task division between Dom0 and the hypervisor.
Last but not least, the book 'The Definitive Guide to the Xen Hypervisor' by David Chisnall comes highly recommended. It covers all these topics and more in a thorough, technical fashion with plenty of code examples.
The Xen wiki and developer mailing lists are also a great resource for understanding Xen.
If you have a more specific question, then I can give you a more specific answer.
Here are few links which will guide you with ZEN Start up.Hope they will be useful.
http://www.howtoforge.com/howtos/virtualization/xen
http://wiki.xen.org/wiki/Category:HowTo
http://wiki.debian.org/Xen
For me, that is the best and more concrete tutorial with examples and step by step to start. I used it when I started.
Then you can read a lot more on Xen documentation itself or some books but as a starting point that allows you to easily install and test Xen, I choose that tutorial from Debian Wiki.
If you just want an overview, you may read this: http://wiki.xenproject.org/wiki/Xen_Project_Beginners_Guide.
This will introduce you to Xen hypervisor, suggest configuration to set up virtual machines, provide information about the networking and finally have details about tools for the management of virtual machines.
This documentation is to get the Xen specifically on ubuntu (Most importantly, it works!)
https://help.ubuntu.com/community/Xen
===
However, if you want to go to the next level and understand the working of Xen; Xen architecture, memory management, device management, CPU scheduling etc., I would recommend reading the book "The Definitive Guide to the Xen Hypervisor".
I have a bunch of R scripts which I am running on a Windows machine and want to ensure that the code remains unread by those not intended to see it. On a Linux box, I could wrap the R code in a bash script #! and make an encrypted (and perhaps even a limited-life) executable shell script. What are my options to do something on similar lines under Windows?
My answer is a bit late, but I believe this is a good question. Unfortunately, I don't believe that there is a solution, or at least an easy one, at the present time.
The difficulty is common because, for most interpreted languages, including R, it is often possible to turn on logging and inspection of all commands being run. This can negate many tricks to obfuscate the code.
For those who prefer to think of code being open == good, one should know that a common reason to obfuscate the code is if one is consulting with a client that hires multiple vendors. It is not uncommon for a client to take scripts from vendor A and ask vendor B why it doesn't work with their system. (This may be done by a low-level IT flunkie, rather than someone responsible for the NDA contracts.) If A & B are competitors, A's code has just been handed to B. When scripts == serious programs, then serious code has been given away.
The ways I've seen this addressed are:
Make a call to a compiled language, and use standard protections available there.
Host the executable on a different server, and use calls to the server to execute the calculations. (In R, there are multiple server-side options.)
Use compiled (preprocessed / bytecode) code within the language.
Option 2 is actually easier and better when the code may be widely distributed, not just for IP reasons. A major advantage is that it lets you upgrade the code without having to go through the pain of a site-wide release process. If new libraries are needed, no problem - update the server.
Option 3 is done in Matlab with .p files, and can be done with py2exe for Python on Windows. In R, the new bytecode compilation may be analogous, but I am not familiar enough with it to address any differences between .Rc files in the R context and .p files in the Matlab context. For more info on the compiler, see: http://www.inside-r.org/r-doc/compiler/compile
Hosting computations on the server is great for working with unsophisticated users, because it is easier to iterate quickly in response to bugs or feature requests. The IP protection is simply a benefit.
This is not a specifically R-oriented strategy. (And it's a bit unclear what your constraints or goals really are anyway.) If you want a cross-platform encryption method, you should look into the open-source program TrueCrypt. It supports creating encrypted files that can be mounted as volumes on any machine that supports the volume formatting method. I have tested this across the Mac PC divide , since the Mac can read FAT files, but have no experience with how it might work across the Linux-PC chasm.
(Their TODO list for Windows includes;"Command line options for volume creation (already implemented in Linux and Mac OS X versions)". So I don't see any clear way to use this from within R without you running the program from the OS.)
I don't think this is possible because the R interpreter has to be able to decrypt and read the code in order to execute it which means that whoever is using that interpreter will also be able to decrypt and read the code.
I am by no means an expert, so I reserve the right to be 100% wrong about that statement.
I believe the best solution is to ensure value comes from the expertise and services provided by your company and it's employers---not from keeping secrets.
Failing that, you could try separating the code into a client/server model. That way the client just sends data and receives results---they never have access to the code that runs on the server.
However, the scientist in me just said "that solution sucks and I would never trust results provided under such conditions".
I would like to develop a Network Inventory application that works on any operating system.
Reports on every possible resource attacehd to a network.
Reports all pertinent details of hardware and software.
Thats (and i hate to use the phrase) my "End Game".
However I am running before i can crawl here.
I have no experience of this type of development, e.g. discovering a computers hardware and software settings.
I've spent almost two weeks googling and come up short! :-(.
So I am turning to you to ask these questions:-
My first step is to find an existing open source project i can incorporate into my own code that extracts the fine grained details i am after, e.g. EVERYTHING there is to know about the hardaware and software on a single machine.
Does this project exist? or do i have to develop that first?
Have i got to write all this in C?
I am guessing getting this information about a computer is going to be easier than for printers, scanners, routers etc... e.g. everything else you would find attached to a network.
Once i have access to a single computers details i then need to investigate how i can traverse an entire newtork of printers, scanners, routers, load balancers, switches, firewalls, workstations, servers, storeage devices, laptops, monitors, the list goes on and on
One problem i have is i dont have a 1000 machine newtork to play on!
Is there any such resource available on theinternet? (is that a silly question?)
Anywho, if you dont ask you wont find out!
One aspect iam really looking forward to finding out how to travers the entire network,
should i be using TCP/IP for this?
Whats a good site, blog, usergorup, book for TCP/IP development?
How do i go about getting through firewalls?
How many questions can i ask in one go? :-)
My previous question on this topic ended up with PYTHON being championed as the language/script to go with to develop this application in.
Having looked at a few PYTHON examples they all seemed to be related to WINDOWS networks
and interrogating Windows Management Instrumentation (WMI). I had the feeling you cant rely on whats in WMI, and even if you can that s no good for UNIX netwrks.
Surely there exist common code for extracting hardware and software details from a computer? Why cant i find it on the internet?
Pease help?
Theres no prizes though :-(
Thanks in advance
I would like to appologise if i have broken forum rules or not tried hard enough on my own before asking for assistance.
I just would like to start moving forward with this as its one of the best projects i have been involved with.
I am inspired by the many differnt number of challenges involved and that if i manage to produce a useful application at the end of it it would hopefully be extremely helpful to many people.
That sit
Thanks in advance
DD
as a software vendor of a discovery solution, I can just say: Respect, that you want to start a new one :-). Just in case you are interested in what it could look like: http://www.jdisc.com
Now to some of our experience:
Programming Language:
I wouldn't write it in C. Use Java or .NET. Those languages have great advantages when it comes to tracking down errors or problems. For instance, in Java (and I guess also in .NET), you can see the stack trace when something is failing. For some pieces of code (e.g. WMI access), you might need to use C++ or C (e.g. access to native APIs from Microsoft). Use a native interface or a COM bridge from Java. In .NET, it should even be easier to access the Windows APIs).
Devices:
well, network printers, router, and switches are actually easier to discover. They usually expose their information via SNMP. SNMP is pretty easy to use and pretty robust. Getting information from Windows (or even Unix) systems is a bit trickier. Protocols can be blocked, misconfigured, messed up... We had cases, where WMI was simply hanging when requesting data from a remote device.
Test Devices:
Since we are also a smaller company, we also do not have 1000 different devices to test with. But, there are some things that might help:
a) For SNMP devices use a SNMP simulator. We use MIMIC 9.0 from Gambit Solutions and we are pretty happy with it. You can import SNMP walks from network devices and simulate the device as if it would be in your network.
b) Secondly, use virtualization whenever possible. With VMware, you can install Windows, Linux, or even Solaris. We also use a project called GNS3 to emulate Cisco Routers, Firewalls or Juniper routers.
c)You can test the rest of the devices only, if you have a customer that helps you with testing and implementing new devices.
This are just some ideas to start with. But I have to tell you, that it is not trivial and it takes a lot of time....
Hope that you got some ideas to start with...
I don't know that it's open source, but we use Spiceworks (http://www.spiceworks.com) here as an IT management platform. You may get some use out of exploring that.
One of the annoying things about Smalltalk is that it (usually) requires its own VCS, due to the way that it manages its source code. Squeak and Gemstone (at least in its GLASS version) have a DVCS called Monticello that works passably well. As near as I can tell, VisualWorks' main VCS, StORE, only works in old-fashioned centralized mode. Is there a system similar to Monticello available for VisualWorks? Alternatively, am I misunderstanding the right way to use StORE?
While Store is not structured to be a distributed tool, it can be used that way. Most of the engineers working on VisualWorks and ObjectStudio use a local repository (in their office; most of the team is geographically distributed). There are merging and replication tools that allows them to synch up their local work with the "repository of record" as required/desired.
Having said that, there's a port of Monticello for VW (and thus, for ObjectStudio as well). The only real issue with it is that after publishing, it doesn't "talk" to the tools (browser) about the publish status. That's because it's mostly use to maintain the Seaside port, and not as a mainline versioning tool for VW.