I've been tasked to develop an OpenCL application for a specific platform, Intel CoffeeLake-H GT2 (UHD Graphics 630). There are two problems for me:
Even having some OpenCL programming experience (not that much though), I wouldn't know where to begin. I have no prior experience with targeting specific hardware before.
The device itself has to be emulated or something, because I don't have it at hand.
Of course, I tried googling information today, but couldn't find anything that could really help me. Guess, it's just because of my lack of experience. So, I'm stumped right now, and asking for help.
It would be really great if I can be helped. Any help would be appreciated. Thanks in advance.
Small note: I'm working on this project under Ubuntu 18.04.
I'm not aware of any emulated environment, and anyway, ultimately nothing replaces access to the target hardware. I see a few workarounds:
Target a similar-enough device. Intel GPUs haven't changed that drastically, so especially if you have an older/lower-spec one around, whatever you end up with should run better on the newer GPU. You can also work with a GPU from another vendor if you have at least sporadic access to a system with an Intel GPU. You don't want to go for too long at a time without testing on your target hardware. (It's generally a good idea to test OpenCL code against different implementations while developing, as it's easy to accidentally rely on implementation-defined or undefined behaviour otherwise.)
Rent a relevant physical device. Places exist that allow you to rent laptops or desktop PCs for a short time period.
Remote access to a target device. Presumably whoever posed the requirement actually has such devices. Ask for remote access to one of them, via the magic of the internet. (RDP, VNC, SSH)
Rent similar hardware in a data centre. There are bare metal hosting companies that rent out physical servers built from commodity hardware. Find one that offers servers with a close enough match to the system you're targeting and rent one there.
As for the skill gap, well, you'll either have to bridge that one yourself by following enough documentation, tutorials, etc. or by finding (hiring…) someone who will give you some degree of hand-holding through the project.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am a Computational Fluid Dynamist (CFD), but I dont know mpi very well.
Since heavy CFD jobs requires Infiniband support, and people say that mvapich is usually much better than other mpi implementations. I want to know is this true? Any one has any real experience and references that I can look at? And how come that mvapich is better than the openmpi, etc.? Is it written by Infiniband company or what?
Thanks a lot!
So the answer is "probably not, and it doesn't matter anyway".
You write your code using the MPI API, and you can always install multiple MPI libraries and test against each, as you might with several LAPACK implementations. If one's consistently faster for your application, use it. But the MPI community is very concerned with performance, and the free competitors are all open source and publish their methods in papers, and all publish lots of benchmarks. The friendly rivalary, combined with the openness, tends to mean that no implementation has significant performance advantages for long.
On our big x86 cluster we've done "real world" and micro benchmarks with MPICH2, OpenMPI, MVAPICH2, and IntelMPI. Amongst the three open-source versions, there was no clear winner; on some cases one would win by 10-20%, on others it would lose by the same amount. On those few occassions where we were interested enough to dig into the details to find out why, it was often just a matter of defaults for things like eager limits or crossover between different collective algorithms, and by setting a couple of environment variables we got performance within noise between them. In other cases, a performance advantage persisted but was either not large enough or not consistent enough for us to investigate further.
(IntelMPI, which costs significant amounts of money, was noticibly and mostly-consistently faster, although what we consider the big win there was substantially improved startup times for very large jobs.)
MVAPICH was one of the first MPI implementations to really go after Infiniband performance, after having lots of experience with Myrinet, and they did have a significant advantage there for quite some time, and there are probably benchmarks in which they still win; but ultimately there was no consistent and important performance win and we went with OpenMPI for our main Open Source MPI option.
I would agree with Jonathan regarding the answer and add a few points from a cluster administration perspective.
As a person who at times dips into cluster administration I would add that tuning InfiniBand on a large cluster is not an easy task. You have to make sure that the OFED stack sits well upon your kernel. That the hardware is not faulty and the switches are working as expected without performance issues in a sustained mode and the application maps correctly onto the InfiniBand topology and lots more.
OpenMPI stack is considerably different from MPICH/MVAPICH. I find that OpenMPI component architecture makes it easier to find and debug issues than the architecture of MPICH/MVAPICH which I find more monolithic.
Speaking of vendors recall that MPICH comes from the MCS department of Argonne.
Update: Since version 3.1 MPICH supports OFED InfiniBand via the ib network module. Since 3.2 MPICH will support also the Mellanox MXM interface.
MVAPICH is built on top of MPICH sources by the people from the department of CS&E at Ohio State.
Many hardware vendors build either on top of MPICH or MVAPICH to provide InfiniBand support for their respective HW. One example is Intel MPI. Another is Voltaire MPI.
OpenMPI is developed by several teams supported by InfiniBand switch vendors like Cisco.
HP MPI used to be another very good MPI implementation for generic clusters that is currently available from Platfrom.
CFL codes don't scale well.
I can't speak directly to MVAPICH2, but I would recommend using whatever MPI is native to your cluster. So if you are using a Cray machine, you would go with Cray's MPI. It works like magic. Using your vendors recommended mpi makes a significant difference.
To directly answer your question, if your message size falls in the short range MVAPICH2 has a sweet spot where it beats OpenMPI. I think your CFL codes may fall in this range. On large clusters I have found that something goes wrong with MVAPICH2 with latency values when operating on over 2k PEs - but people don't run CFL on 2k PEs.
Ultimately, there is sufficient motivation to test this theory. Which code are you running OpenFOAM, Fluent?
As a modern large company, is one ERP system better than hundreds of highly specialized applications which are service oriented? To provide a little bit of background, we are providing consulting for a client who wants to invest their resources in a monolithic ERP system which will manage everything! What are the pro's and con's of this approach?
As an application developer, I tend to believe that specialized well written and managed software packages tied together by a service architecture would out perform a monolithic approach.
What do you think?
As an application developer, I tend to
believe that specialized well written
and managed software packages tied
together by a service architecture
would out perform a monolithic
approach.
Maybe, but getting support for one system from one party is easier than getting support from multiple parties and making sure that integration works and keeps on working.
I think a more important question is whether to pick a general ERP or a custom fitted one. Whether the architecture is service oriented or monolithic is maybe is related, but also general ERP systems can be service oriented.
This almost feels like a traditional question on buy vs. build. I will try to lay out
some importan points.
If you clients has deep pockets only then can they viably maintain the high total
cost of ownership and complexity associated with developing and
maintaining custom-designed applications.
Off-the-shelf ERP solutions integrate the best business practices from a variety of
industries and incorporate these best business practices into your
client's operations which ultimately translates into bottom-line improvements.
Custom-designed applications provide the desired degree of functionality,
but their size and complexity require lengthy design, development, and
implementation efforts.
A good example that I can think of is Microsoft. Microsoft spent 10 months and $25 million installing SAP R/3 to replace a
tangle of 33 financial-tracking systems in 26 subsidiaries. As a result of the
implementation, Microsoft estimates annual savings at $18 million,
leading Bill Gates to call SAP "an incredible success story."
Hope this helps you think more broadly from all angles.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
One thing that I keep hearing in reference to ASP.NET and MSFT technologies is that they cost money to use. Often when they are being compared to open source languages someone will mention that one factor in favor of open source is that it's free (to an extent). My question is, when does ASP.NET actually cost money to use in terms of using the proprietary technology?
Understandably there are the hosting fees, but I'm curious about the fees outside of these hosting fees. I'm especially curious about this as it relates one-person smaller-site development (non-team/large enterprise). Any help is appreciated.
(edits)
Some excellent answers. Much appreciated
The projects that I'm looking to use the technologies for would be personal sites and very small business sites (1 or 2). The intent would of course be that these projects get much bigger. It seems that for commercial production, fees will apply. What about just basic dynamic "shared hosting" sites that provide information?
You have to measure many things when you determine cost. We recently went through an evaluation of platform choice by an outside vendor, and the recommendation is that we stay with a Microsoft.NET platform. Why? For us, the reason is that once you get to an enterprise-level product suite, the difference is not as big as people would like you to believe.
Purchasing Microsoft products is a sound choice. The initial cost might seem high, but keep in mind if you get Software Assurance on your purchase (Visual Studio, for example) you are entitled to free upgrades as long as you keep you SA current - and it is at a fraction of the cost of a repurchase. Many people think you need to buy the full retail version every time, and that is just not true. Work with a larger vendor, like CDW, to help with licensing questions. They got someone from Microsoft's Licensing Division on the phone with us and helped us choose what was right. Not high-pressured at all. They actually talked us down on some of the things we thought we needed.
MSDN subscriptions are great. I have one through my employer, but also used to maintain one personally. If you are a contractor/self-employed, it is an operational expense. Like buying full products, renewing a MSDN subscription is very inexpensive compared to a purchase, and especially considering what you get. The licensing within MSDN is rather generous, and since you are a one-person shop, if I read that correctly, one MSDN is more than enough for your non-production needs. Plus, the bundled Support Incidents are a nice plus, as well.
There are many versions of Visual Studio, from the Express Editions all the way up to the Team Editions. For example, we are rolling out Team Foundation Server right now, so our costs are obviously higher. For a startup or small shop, there are TFS hosting partners and you can get Team Explorer for free. Or you can mix and match, using Visual Studio for development and something like VSS, SVN, or countless other version control products out there.
Just because someone "goes open source", that does not mean that it is free. Yes, the platform choice might be free, and the tools might be free, but there is a definite chance that you will need a commercial library or component some day. Plus, nothing prevents you from going with Open Source products with Microsoft, either. There are many open source projects written in .NET that can be leveraged with your solutions, and Microsoft is becoming a lot more transparent. We are working on a very large, enterprise solution right now and we are using only one "commercial" product, outside of our development tooling. There is a lot of Open Source usage, and a lot of implementations cobbled from community musings and examples.
The one thing that often goes unmentioned is the human cost that goes into these decisions. Microsoft is hated by many and their solutions might not be the fastest or most robust (although I will take IIS7 on W2K8 over any other web server configuration any day) they are focusing on making people more productive at what they can do. You aren't just buying products with them, you are actually buying productivity. As someone who has worked in a few Open Source shops, I am very appreciative of all the things that they have gotten right and understand that free does not always mean better.
I have a "one-person" side business and I really recommend looking into an MSDN subscription. It will give you access to tools and technologies that you would not otherwise be able to get your hands on without going a la carte in a retail route. Talk with someone, like a CDW, to help you figure out your licensing needs. If it works out, definitely try that route. You can cover all your in-house needs in a one person shop with an MSDN subscription, most likely (for example, a lot of the products are available to install to you (as a user) up to ten times as long as the machines that they are installed on are "yours" and non-production. There are exceptions to that, but not many.)
If that does not work, try the free route. You can definitely use Mono for .NET Development, as well as the Express Editions. I know a few C# developers who swear by Mono and could not be happier.
Best of luck to you!
Often when they are being compared to open source languages someone will mention that one factor in favor of open source is that it's free (to an extent). My question is, when does ASP.NET actually cost money to use in terms of using the proprietary technology?
Usually when people refer to "cost" in the way you described, they're implicitly referring to TCO, or total cost of ownership. The cost is not an explicit cost in that you've paid for something directly, but rather the long-term price of using something over its lifetime.
For example, even if a particular proprietary technology is free, it may be more difficult to hire and find people who know about it to work on your project. Consequently, if it is less popular than some open-source equivalent, you may wind up paying more for the same amount of labor because appropriately talented staff will be harder to find and in higher demand.
Conversely, if an open-source product is free but has low mindshare or performs poorly, it may well be worth it to pay for an expensive, closed-source proprietary solution rather than having to learn the idiosyncracies of the open-source version.
Naturally, there is some controversy surrounding just how to measure TCO, with both camps having some valid points.
.NET is free
C# compilers are free
Certain versions of Visual Studio are free, and you don't actually need it to write for .NET anyway (though it really, really helps!)
There are many free online resources for learning .NET, such as http://asp.net
In short, there's no real cost to using ASP.NET other than the hosting fee of the website or options you might buy to make things easier (better versions of Visual Studio.)
There's more of an ideological divide, with open source guys on one side being pretty anti-microsoft and so claiming it's high cost to use. I wouldn't worry about them. ;)
There are a couple of good answers already, but I'd like to add "it depends".
joseph.ferris obviously works in a large organization, where the cost of switching platforms is going to be very, very expensive, so the cost of paying the licensing costs is much less that the cost of switching. Take a look at Jonathan Schwartz's blog entry for Mar. 11, especially the section titled "When Free is too Expensive" for another reason to go with fully-supported infrastructure.
But consider a couple of other scenarios.
First, there's the hobbyist, which is what you seem to be addressing - you want to play around with the technology, and maybe put up a website or three. There aren't any issues with privacy or scalability, so you can deploy your application on an inexpensive shared hosting solution. In this case, costs are pretty much irrelevant - whatever platform you pick, you can get free tools to get you started. Remember kids, the first hit is always free.
For a startup, things are a bit different. If the goal is to build a large website, the potential licensing costs can be daunting - it's probably going to a lot cheaper to go with open source. In addition to the production environment, you need to pay for development environments, testing, etc. Even for a small company, licenses may be more than they have in the budget - a single Windows 2003 server Enterprise license lists for $4k. If you're trying to break into a competitive environment and compete on price, this alone may make you uncompetitive. I have seen situations where a Windows-based solution (server, database, and custom development coupled with a content management system) is two or three times the price of an open source solution.
I know that it has been answered, but I will put my 2 cents. Why are you wondering about the cost of ASP.net? In my opinion, the choice of technology in your case (1-2 ppl development freelancer team) should be governed by technology familiarity. If you are an ASP.net expert, the expense of buying the products and MSDN subscription is well-worth it, because it's your primary language of choice that you know well, hence the projects that you implement, will be done better and faster, so it makes sense to stay with it.
However, if you happen to know another technology just as well and you are comfortable that you can deliver a robust product on-time with it, it may be worth it to go low cost. As a contractor, the main objective is to not lose time/money hence you pick a technology that balances your expenses and time spent learning it. In other words, if you are a Java expert, there is no point of paying for asp.net. If you know asp,net well already, then sure, stay with it.
The clients rarely care whether you used Ruby, PHP, Python, Java or ASP.net. They care about time lines, their cost and quality.
I find that it does not cost much money to use. It does infact cost a pretty penny to get windows based hosting. Visual studio is also expensive. After those, though, not many expenses are encountered.
If you want to use the more professional versions of Visual Studio to develop your applications: you will need to pay for that.
Also, there are a lot of commercial components available on the market. These will save you time or improve your product, but at a cost.
For open source, there are also a lot of components, but in this scene most is free/open.
I'm starting to build a community website from the site up and my web framework will be Asp.net and Mysql.
I want to start planning some scalability into the infrastructure early because I'm anticipating high traffic when the site goes live.
Are there any case studies which you recommend reading where asp.net or mysql has been scaled and which demonstrates good scaling techniques?
I think it could be a challenge to find reference materials for that particular combination. Many .NET shops stick to SQL Server, and fewer use MySQL (at least at scale).
In general it would be appropriate to:
Follow general .NET practices for scalability. Weed out what is not appropriate for you.
Learn about database performance and implications of various design strategies such as denormalisation (when and why).
Consider out-of-process caching like memcached.
Review books on MySQL performance. Most of these are focused on UNIX platforms. Windows users may have problems applying some of these practices.
Read up on how other people are scaling their sites (Building Scalable Sites and The Art of Capacity Planning)
Consider how you might optimise your web design to be more scalable. Are you using AJAX? Work out what the impact of excessive polling may be etc.
Learn how to measure the performance of your application and database (starting points ASP.NET and MySQL).
Develop a plan for scaling your architecture (1 server to 2 servers, to multiple servers etc) so that you have some frame of reference for making decisions about building things in your system.
I only know of one really good resource to read case studies about scalability techniques and I am really surprised no one has mentioned it. High Scalability
There is so many examples of "out of the box" thinking that and different techniques for scaling that I think it makes a good read for anyone who is interested in the topic.
BrianLy said it best here:
"Develop a plan for scaling your
architecture (1 server to 2 servers,
to multiple servers etc) so that you
have some frame of reference for
making decisions about building things
in your system."
As a forum I frequent says, 'quoted for truth'. All of his points are excellent, but this one is a key point that many people overlook. It doesn't matter how scalable your code and database are if you are running on a creaky old server. The hardware may not be as important as your code, improving it beyond a certain point will give diminishing returns VERY quickly, but do NOT forget to get your hardware to that point. If you have crap hardware, or even good hardware but not enough of it, your site will bomb out.
For mysql scaling, you may find this interesting: danga livejournal