While developing products, we often need to create proprietary tools to test some of their unique features or diagnose problems. In fact the tools can be at lest as interesting as the products themselves, and some of our internal groups have asked for copies of them.
So, aside from the obvious business-driven rules (e.g. don't retrieve sensitive data), what do you differently when you build personal or internal tools, as opposed to for-sale products, and why?
What's more (or less) important to you in internal tools, and do you consider overall value to the company when you build them?
Thanks for your thoughts!
First, internal tools are always developed quick and dirty. Almost no testing - it just has to do the work.
UI is not as important as with a customer-facing app.
Internal tool can use internal/private/proprietary knowledge of the products and frameworks they test. For example, our last product bypassed part of our published API and used a non-documented web service call to achieve better results.
This is an important point,but a losing battle: NEVER EVER leave internal tools with a customer.
As a consultant, I sometimes had to use and even develop those tools in the field. I try to hide it from my clients, but from time to time, they demand I leave the tool with them (or worse, call the sales rep and ask for that "magic tool"). You don't want customers judging your entire company's production level based on tools build according to points 1-3.
From an engineering perspective, I wouldn't do anything differently:
Both internal and for-sale tools need to be well-written and well-documented
Both need to be created given a set of requirements, deadlines, budgetary restraints, etc.
Both need to be tested or validated
The one big difference I see would apply to the for-sale products as opposed to the internal tools: for-sale products need marketing, support, etc that internal tools can do without.
Additionally, since internal tools will be used in a somewhat more controlled environment, they don't need to be tested against different computer systems, Internet browsers, etc.
The biggest difference:
It's with the personal and internal tools that you can be more free to try out a new technology, the latest fashion. You can take risks that you wouldn't take with the application that you are actually shipping to customers.
Since the diagnostics I build are usually very special-purpose, I tend to provide more options and built-in examples than I would for customer-facing products. In other words, I assume the user is more familiar with the technology than a customer would generally be, and I provide more ability to tweak the way the tool operates without worrying that it might overwhelm the user. But I also try to make it satisfy 80% of the use cases without much "help" from the user.
Related
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Our product is a web based course managemant system. We have 10+ clients and in future we may get more clients. (Asp.net,SQL Server)
Currently if one of our customers need extra functionality or customised business logic, we will change the db schema and code to meet the needs.
(we only have one branch code base and one database schema)
To make the change wont affect each others route, we use a client flag, which defined in a web config file, thus those extra fields and biz logic only applied to a particular customer's system.
if(ClientId = 'ABC')
{
//DO ABC Stuff
}
else
{
//Normal Route
}
One of our senior colleagues said, in this way, small company like us can save resources on supporting multiple resources.
But what I feel is, this strategy makes our code and database even harder to maintain.
Anyone there crossed similar situation? How do you handle that?
Update:If this is not a right question for SO, can someone move this question to a proper stackexchange site?
Update2:
you are right. The code is becoming smelly now, and I quite sure will be a nightmare sooner or later. Our company is doing the product and to save the effort, later products for other customers are based on the previous one. I know the ideal way is seperate the #e-j-brennan dev teams into 2 parts. One team works on core product and made it highly customisable, and team two works on customising for a particular client. However if since our company is so small, it is really a dilemma situation. :(
I think you need to decide if you sell custom software, that you tailor for each client, or 'off-the-shelf' software that is one-size-fits-all (and maybe customizable thru functionality you provide).
When you only have a handful of clients like you do now, you can get away with what you are doing, but I can almost guarantee that if you continue down this road, and your client base increases and the amount of client-specific customization's increases as well, you will have a nightmare on hand; I've been thru this many times for multiple clients, and it always ends the same way. It all is manageable until it is not, and then it is a royal pain-in-the-neck that could make your life very difficult indeed.
If you decide you are a custom company, and want to have multiple versions of the software and database, that is fine, just make sure you charge the full cost for it - i.e. factor in that you may need to maintain multiple levels of source code and databases and factor in that upgrades are going to take many multiples of effort to rollout as you will need to test each client's code base.
If you decide you want to be an 'off-the-shelf' type of product, then your best bet is provide the ability for each client to customize their experience, without the need for code changes - i.e. built in the customization capability thru config screens and tables that control how things work - but everyone will still use the same underlying code and database. Much more work upfront, but saves you boatloads of time down the road.
I have also been in your position, and I agree it is a difficult one. In my case, I was building custom single-product sites for clients. While each site followed a similar layout and workflow, there had to be enough flexibility for each to have a wholly custom design, custom rules around shipping and coupons, and different merchant gateways and configurations.
After some years, we did end up with something maintainable. First, we created libraries to house all of our common code and put those libraries into a TFS project simply called Common. Then, we created a new TFS project for each site (not client, as many clients had multiple products/sites) and branched the applicable projects into them from Common. Next, we created a VS Template project that contained a skeleton of the site, including "design-less" views, controllers, and their action methods (remember, each site had the same basic flow). Also, each site ran on its own database, which was cloned from an otherwise unused and mostly empty Template DB.
With each site running on its own branch and DB, modifications could be made to the original flow and design that was installed by the template (which would never need to be merged back in) without affecting any other site. For customizing business methods, like shipping calculations, we could create a subclass of the common class and override where needed. Part of what enabled this was converting all our code to use Dependency Injection. Specifically, each Controller had injected Services, and each Service had injected Repositories. Merchant Processing was also coded to an interface and injected. Also worth mentioning is that this allowed us to hard-code all of the upsell logic for each site (you bought product X, so we recommend Y), which was much easier to create and maintain compared to defining complex configuration rules in our old upsell rule engine. I don't know if you have anything like that...
Sometimes we would want to make a change to the Common code itself, which was usually prompted by a specific need for a specific site. In that case, we'd make the change on that branch, merge it to Common, and then merge it to the other sites at our convenience (great for "breaking" changes or changes that also required a change to the DB). Similarly for DB changes, we would update the Template DB and then write a little script to update the other site DBs with the same schema changes ( still had to be smart and careful about it).
An added benefit was that we also created Mock repositories that would be used/injected in a "Design" build configuration, which enabled the designers to jump around the application and work on screens without literally submitting themselves to the workflow. It also allowed them to start working on a site before there was anything done on the back-end, which was very important for those anxious clients who need to "see something".
10+ clients is definitely not a small number with what you're talking about. Three was pain enough for me. We had over 30 sites running at one time, maintained by three developers and two designers.
Finally, I know it's outside the scope of your question and a bit presumptuous, but getting "final" client sign-off on design before the designers actually went about implementing it (and before devs did their thing) also saved us a lot of costly rework. I know no design is final, but increasing efficiency on the implementation end gave the clients less time to change their minds about the design they approved.
I hope that at least gives you some approaches to think about.
People working with systems that have to change or be customized, have developed patterns to handle such concerns.
You should definitely start by reading a good book on Inversion of Control. In short, you can build your systems by defining building blocks (contracts, expressed as interfaces) and provide multiple implementations. There are multiple benefits of such approach but to mention just two:
- you can handle customizations by providing diffent implementations of the same interfaces
- you can reconfigure your application statically or dynamically but both approaches are far more clean than your "if"
When it comes to the data layer, study the repository pattern. It helps to organize the data access in a way that you can switch between different providers. It fits great wiht ioc.
And just a technical tip - nhibernate supports dynamic properties. You just provide additional columns in the mapping and nh is able to support it from the same code base. This way you can target different databases with slightly different db schemas.
I've always been of the opinion an internal development group should really only be building/maintaining three applications.
An internal composite/pluggable/extendable application.
The company website.
(Optional) A mobile version of #1 for field employees.
I'm a consultant, and everywhere I go, my clients have dozens of one-off applications in the web and on the desktop for every need no matter how related to the others. Someone comes to IT and says "I need this", and IT developers turn around and write another one-off ASP.NET application, or another WinForms app.
What's your opionion? Should I embrace the "as many apps as we want/need" movement? I assume it's common; but is it sensible?
EDIT:
A colleague pointed out that it depends on the focus of the development - are you making apps or are you making a system? I guess to me, internal development is about making a system; development of shippable software products, like MS Word, iTunes, and Photoshop, is about making apps.
All of them?
Wow do I ever agree with you. The problem is that many one-off applications will (at some point) each have many one-off maintenance requests. Anything from business rule updates to requests for new reports. At some point the ratio of apps that need to be maintained to available development staff is going to be stretched/taxed.
From my perhaps (limited?) vantage point, I'm starting to think #1 and #3 could be boiled down to Sharepoint. Most one-off applications where I work (a large 500+ attorney law firm) consist of one or more of the following:
A wiki
A blog
Some sort of list (or lists joined together in some type of relationship), which can be sorted and arranged in different ways.
A report (either a Sharepoint data view or a SQL Server Report work just fine)
Or, the user just wants to "make a web page" and add content to it. But only they should be able to edit it. Except when they're out of the office, and then, etc...
Try to build any one of the above using [name your technology], and you've got lots of maintenance cycles to look forward to (versus a relatively minor Sharepoint change).
If I could restate what I think is your point: why not put most of your dev cycles to work improving and maintaining a single application that can support most of your business' one-off needs, rather than cranking out an unending stream of smallish speciality apps?
This question depends on so many things and is subjective besides. I've worked at companies that have needed several different apps because we do business in discreet silos. In that case, an internal group may not build and maintain apps, but may build several, with another group that is responsible for maintenance.
Also, what do you mean by "app"? If you broaden the term enough, then you could say "it's all just one big app".
In short, I think the main consideration is the capacity of the group and what business needs are.
I think there should be internal development teams that each has a system which may contain multiple applications within it. To take a few examples of what I mean by systems:
ERP - If you are a manufacturer of products, you may need a system to keep track of inventory, accounting of books and money, and other planning elements. There are a wide range of scales of such systems but I suspect in most cases there is some customization done and that is where a team is used and may end up just doing that over and over if the company is successful and a new system is needed to replace the previous one as these can take years to get fully up and running. The application for the shop floor is likely not the same one as what the CFO needs in order to write the quarterly earnings numbers to give two examples here.
CRM - How about tracking all customer interactions within an organization that can be useful for sales and marketing departments? Again, there are many different solutions and generally there is customizations done which is another team. The sales team may have one view of the data but if there is a support arm to the company they may want different data about a customer to help them.
CMS - Now, here I can see your three applications making sense, but note what else there is beyond simple content.
I don't think I'd want to work where everything is a home grown solution and there is no outside code used at all. Lots of code out there can be used in rather good ways such as tools but also components like DB servers or development IDEs.
So what's the alternative to several one-off applications? One super-huge application that runs everything and everything? That seems even worse to me...
I have been tasked with automating some of the paper forms in HR. This might turn into "automate all forms" eventually, so I want to approach this in a way which will be best for the long term and will be a good framework as this project grows.
The first things that come to mind were:
-InfoPath/SharePoint (We currently don't use SharePoint now, and wouldn't be an option for the next two years.)
-Workflow Foundation (I've looked into this and does not seem too attractive or appropriate)
Option I'm considering at this point:
-Custom ASP.NET (VB.NET) & SQL Server, which is what my team mostly writes their apps with.
-Leverage Infopath for creating the forms electronically. Wondering if there is a good approach to integrating this with a custom built ASP.NET app.
-Considering creating the app as an MVC web app.
My question is this:
-Are there other options I might want to consider?
-Are there any starter kits or VB.NET based open source projects there which would be a starting point or could be used as a good reference. Here I'm mostly concerned with the workflow processing.
-Any comnments from those who have gone down this path?
This is going to sound really dumb, but in my many years of helping companies automate paper form-based processes is to understand the process first. You will most likely find that no single person understands the whole thing. You will need to role-play the many paths thru the process to get your head around it. And once you present your findings, everyone will be shocked because they had no idea it was that complex. Use that as an opportunity to streamline.
Automating a broken process only makes it screw up faster and tell a lot of people.
As far as tools, my experience dates me but try to go with something with these properties:
EASY to change. You WILL be changing it. So don't hard-code anything.
Possible revision control - changes to a process may or may not affect documents already in route?
Visual workflow editing. Everyone wants this but they'll all ask you to drive it. Still, nice tools.
Not sure if this helps or not - but 80% of success in automating processes is not technology.
This is slightly off topic, but related - defect tracking systems generally have workflow engines/state. (In fact, I think Joel or some other FC employee posted something about using FB for managing the initial emails and resume process)
I second the other advice about modeling the workflow before doing any coding or technology choices. You will also want this to be flexible.
as n8owl reminded us, automating a mess yields an automated mess - which is not an improvement. Many paper-forms systems have evolved over decades and can be quite redundant and unruly. Some may view "messing with the forms" as a violation of their personal fiefdoms, so watch your back ;-)
model the workflow in terms of the forms used by whom in what roles for what purposes; this documents the current process as a baseline. Get estimates of how long each step takes, both in terms of man-hours and calendar time
understand the workflow in terms of the information gathered, generated, and transmitted
consolidate the information on the forms into a new set of forms for minimal workflow
be prepared to be told "This is the way we've always done it and we're not going to change", and to gently (a) validate their feelings, (b) explain how less work is more efficient, and (c) show concrete benefits [vs.the baseline from step 1]
soft-code when possible; use processing rules when possible; web services and html forms (esp. w/jquery) will go a long way if you have an intranet
beware of canned packages (including sharepoint) unless you are absolutely certain they encompass your organization's current and future needs
good luck!
--S
I detect here a general tone of caution with regards to a workflow based approach and must agree. Be advised about the caveats of most workflow technologies which sacrifice usability for flexibility.
I was using a CASE called MAGIC for a system I'm developing, I've never used this kind of tool before and at first sight I liked, a month later I had a lot of the application generated, I felt very productive and ... I would say ... satisfied.
In some way a felt uncomfortable, cause, there is no code and everything I was used to, but in the other hand I could speed up my developing. The fact is that eventually I returned to use C# because I find it more flexible to develop, I can make unit testing, use CVS, I have access to more resources and basically I had "all the control". I felt that this tool didn't give me confidence and I thought that later in the project I could not manage it due to its forced established rules of development. And also a lot of things like sending emails, using my own controls, and other things had their complication, it seemed that at some point it was not going to be as easy as initially I thought and as initially the product claims. This reminds me a very nice article called "No Silver Bullet".
This CASE had its advantages but on the other hand it doesn't have resources you can consult and actually the license and certification are very expensive. For me another dissapointing thing is that because of its simplistic approach for development I felt scared on first hand cause of my unexperience on these kind of tools and second cause I thought that if I continued using it maybe it would have turned to be a complex monster that I could not manage later in the project.
I think it's good to use these kind of solutions to speed up things but I wonder, why aren't these programs as popular as VS.Net, J2EE, Ruby, Python, etc. if they claim to enhance productivity better than the tools I've pointed?
We use a CASE tool at my current company for code generation and we are trying to move away from it.
The benefits that it brings - a graphical representation of the code making components 'easier' to pick up for new developers - are outweighed by the disadvantges in my opinion.
Those main disadvantages are:
We cannot do automatic merges, making it close to impossible for parallel development on one component.
Developers get dependant on the tool and 'forget' how to handcode.
Just a couple questions for you:
How much productivity do you gain compared to the control that you use?
How testable and reliant is the code you create?
How well can you implement a new pattern into your design?
I can't imagine that there is a CASE out there that I could write a test first and then use a CASE to generate the code I need. I'd rather stick to resharper which can easily do my mundane tasks and retain full control of my code.
The project I'm on originally went w/ the Oracle Development Suite to put together a web application.
Over time (5+ years), customer requirements became more complex than originally anticipated, and the screens were not easily maintainable. So, the team informally decided to start doing custom (hand coded) screens in web PL/SQL, instead of generating them using the Oracle Development Suite CASE tools (Oracle Designer).
The Oracle Report Builder component of the Development Suite is still being used by the team, as it seems to "get the job done" in a timely fashion. In general, the developers using the Report Builder tool are not very comfortable coding.
In this case, it seems that the productivity aspect of such CASE tools is heavily dependent on customer requirements and developer skill sets/training/background.
Unfortunaly the Magic tool doesn't generates code and also it can't implement a design pattern. I don't have control over the code cause as i stated before it doesn't have code to modify. Te bottom line is that it can speed up productivity in some way but it has the impossibility to user CVS, patterns also and I can't control all the details.
I agree with gary when he says "it seems that the productivity aspect of such CASE tools is heavily dependent on customer requirements and developer skill sets/training/background" but also I can't agree more with Klelky;
Those main disadvantages are:
1. We cannot do automatic merges, making it close to impossible for parallel development on one component.
2.Developers get dependant on the tool and 'forget' how to handcode.
Thanks
I have the task of developing an online examination software for a small university, I need to implement measures to prevent cheating...
What are your ideas on how to do this?
I would like to possibly disable all IE / firefox tabs, or some how log internet activity so I know if they are googling anwsers...is there any realistic way to do such things from a flex / web application?
Simply put, no there is no realistic way to accomplish this if it is an online exam (assuming they are using their own computers to take the exam).
Is opening the browser window full screen an option? You could possibly also check for the window losing focus and start a timer that stops the test after some small period of time.
#Chuck - a good idea.
If the test was created in Flash/Flex, you could force the user to make the application fullscreen in order to start the test (fullscreen mode has to be user-initiated). Then, you can listen for the Flash events dispatched when flash exits fullscreen mode and take whatever appropriate action you want (end the test, penalize the user, etc.).
Flash/Flex fullscreen event info.
blog.flexexamples.com has an example fo creating a fullscreen-capable app.
Random questions and large banks of questions help. Randomizing even the same question (say changing the numbers, and calculating the result) helps too. None of these will prevent cheating though.
In the first case, if the pool is large enough, so that no two students get the same question, all that means is that students will compile a list of questions over the course of several semesters. (It is also a ton of work for the professors to come up with so many questions, I've had to do it as a TA it is not fun.)
In the second case, all you need is one smart student to solve the general case, and all the rest just take that answer and plug in the values.
Online review systems work well with either of these strategies (no benefit in cheating.) Online tests? They won't work.
Finally, as for preventing googling... good luck. Even if your application could completely lock down the machine. The user could always run a VM or a second machine and do whatever they want.
My school has always had a download link for the Lockdown browser, but I've never taken a course that required it. You can probably force the student to use it with a user agent check, but it could probably be spoofed with some effort.
Proctored tests are the only way to prevent someone from cheating. All the other methods might make it hard enough to not be worth the effort for most, but don't discount the fact that certain types of people will work twice as hard to cheat than it would have taken them to study honestly.
Since you can't block them from using google, you've got to make sure they don't have time to google. Put the questions in images so they can't copy and paste (randomize the image names each time they are displayed).
Make the question longer (100 words or more) and you will find that people would rather answer the question than retype the whole thing in google.
Give them a very short time. like 30-45 seconds. Time to read the question, think for a moment, and click either A, B, C, D, E,
(having just graduated from CSUN I can tell you scantron tests work.)
For essay questions? do a reverse google lookup (meaning put their answer into google as soon as they click submit) and see if you get exact matches. If so, you know what to do.
Will they always take the test on test machines, or will they be able to take the test from any machine on the network? If it will be specific machines, just use the hosts file to prevent them from getting out to the web.
If it is any machine, then I would look at having the testing backend change the firewall rules for the machine the test is running on so the machine cannot get out to the interwebs.
I'd probably implement a simple winforms (or WPF) app that hosts a browser control in it -- which is locked in to your site. Then you can remove links to browsers and lock down the workstations so that all they can open is your app.
This assumes you have control over the workstations on which the students are taking the tests, of course.
As a teacher, I can tell you the single best way would be to have human review of the answers. A person can sense copy/paste or an answer that doesn't make sense given the context of the course, expected knowledge level of the students, content of the textbook, etc, etc, etc.
A computer can do things like check for statistical similarity of answers, but you really need a person for final review (or, alternatively, build a massive statistical-processing, AI stack that will cost 10x the cost of human review and won't be as good ;-))
No, browsers are designed to limit the amount of damage a website or application can do to the system. You might be able to accomplish your goals through Java, an activex control, or a custom plugin, but other than that you aren't going to be able to 'watch' what they're doing on their system, much less control it. (Think if you could! I could put a spy on this webpage, and if you have it open I get to see what other websites you have open?)
Even if you could do this, using a browser inside a VM would give them the ability to use one computer to browse during the test, and if you could fix that they could simply use a library computer with their laptop next to it, or read things from a book.
The reality is that such unmonitored tests either have to be considered "open book" or "honor" tests. You must design the test questions in such a manner that references won't help solve the problems, which also means that each student needs to get a slightly different test so there is no way for them to collude and generate a key.
You have to develop an application that runs on their computer, but even then you can't solve the VM problem easily, and cannot solve the side by side computers or book problem at all.
-Adam
Randomize questions, ask a random set of questions from a large bank... time the answers...
Unless you mean hacking your site, which is a different question.
Short of having the application run completely on the user's machine, I do not believe there is a way to make sure they are not google-ing the answers. Even then it would be difficult to check for all possible loop-holes.
I have taken classes that used web based quiz software and used to work for a small college as well. For basic cheating prevention I would say randomize the questions.
Try adding SMS messages into the mix.
I agree with Adam, that even with the limitations that I suggested, it would still be trivial to cheat. Those were just "best effort" suggestions.
Your only hopes are a strong school honor code and human proctoring of the room where the test is being given.
As many other posters have said, you can't control the student's computer, and you certainly can't keep them from using a second computer or an iPhone along side the one being used for the test -- note that an iPhone (or other cellular device) can bypass any DNS or firewall on the network, since it uses the cellular provider's network, not the college's.
Good luck; you're going to need it.
Ban them from using any wireless device or laptop and keylog the machines?
You could enforce a small time window during which the test is available. This could reduce the chance that a student who knows the answers will be free to help one who doesn't (since they both need to be taking the test at the same time).
If it's math-related, use different numbers for different students. In general, try to have different questions for different copies of the test.
If you get to design the entire course: try to have some online homeworks as well, so that you can build a profile for each student, such as a statistical analysis of how often they use certain common words and punctuations. Some students use semi-colons often; others never, for example. When they take the test you get a good idea of whether or not it's really them typing.
You could also ask a couple questions you know they don't know. For example, list 10 questions and say they must answer any 6 out of the 10. But make 3 of the questions based on materials not taught in class. If they choose 2 or 3 of these, you have good reason to be suspicious.
Finally, use an algorithm to compare for similar answers. Do a simple hash to get rid of small changes. For example, hash an answer to a list of lower-cased 3-grams (3 words in a row), alphabetize it, and then look for many collisions between different users. This may sound like an obvious technique, but as a teacher I can assure you this will catch a surprising number of cheaters.
Sadly, the real trouble is to actually enforce punishment against cheaters. At the colleges where I have taught, if a student objects to your punishment (such as flunking them on the test in question), the administration will usually give the student something back, such as a positive grade change. I guess this is because the student('s parents) have paid the university a lot of money, but it is still very frustrating as a teacher.
The full screen suggestions are quite limited in their effectiveness as the user can always use a second computer or w/ multi monitor a second screen to perform their lookups. In the end it is probably better to just assume the students are going to cheat and then not count online tests for anything important.
If the tests are helpful for the students they will then do better on the final / mid term exams that are proctored in a controlled setting. Otherwise, why have them in the first place...
Make the questions and answers jpeg images so that you cannot copy and paste blocks of text into a search engine or IDE (if it is a coding test). This combined with a tight time limit to answer each question, say three minutes, makes it much harder to cheat.
I second what Guy said. We also created a Flex based examination system which was hosted in a custom browser built in .NET. The custom browser launched fullscreen, all toolbars were hidden and shortcuts were disabled.
Here is tutorial on how to create a custom browser with C# and VB.NET.
This will solve your problem. http://www.neuber.com/usermonitor/index.html
This will allow you to view the student's browser history during and after the test as well as look in on their screen during the test. Any urls visited during test time will be logged, so you can show them the log when you put a big F on their report card. :)
No one can stop people from cheating, but everyone can receive different questions altogether.
I prefer you buy available online scripts in market as starting point for it. This will save you time, cost and testing efforts.
Below is one of the fine scripts that I worked with and it worked like charm. Using this as base I developed a online testing portal of over 1000 users using computer adaptive test.
http://codecanyon.net/item/online-skills-assessment/9379895
It is a good starting point for people looking to develop Online Exam System.
I customized the script with the help of their support.