Download a file in D - networking

How do I download a file in D? I have checked out the standard library, and the sample. I would rather use phobos with the newest dmd2 than tango. All I need to do is download a file (hopefully using std.socket and std.socketstream). Could also use etc.c.curl.

etc.c.curl provides the C bindings for curl, so you could use that. That's really the only way that I know of to do it using Phobos at the moment, unless you want to do it with std.socket and handle the HTTP requests and responses yourself (which I assume that you don't really want to do).
However, a D wrapper for the C curl bindings is currently in review in the digitalmars.D newsgroup, which would give you a D API for interacting with curl. Assuming that it passes review (which it probably will, though it may change a fair bit during the review process), it'll end up in Phobos. Once it's merged in, it'll be in the following release. So, it'll probably be in either 2.055 or 2.056, depending on when 2.055 gets released.
Until then, however, you're pretty much going to need to either use the C bindings or download the D curl wrapper currently under review. You can find the documentation here and the code here if you want to try it out. If you do that however, it would be much appreciated if you chimed in on the review in the newsgroup to give feedback on it so that it can be appropriately ironed out and improved prior to inclusion in Phobos.

Related

Static data storage on server-side

Why some data on server-side are still stored in DBC files, not in SQL-DB? In particular - spells (spells.dbc). What for?
We have a lot of bugs in spells and it's very hard to understand what's wrong with spell, but it's harder to find it spell...
Spells, Talents, achievements, etc... Are mostly found in DBC files because that is the way Blizzard did it back in the day. It's true that in 2019 this is a pretty outdated way to work indeed. Databases are getting stronger and more versatile and having hard-coded data is proving to be hard to work with. Hell, DBCs aren't really that heavy anyways and the reason why we haven't made this change yet is that... We have no other reason other than it being a task that takes a bit of time and It is monotonous to do.
We are aware that Trinity core has already made this change but they have far more contributors than we do if that serves as an excuse!
Nonetheless, this is already in our to-do list if you check the issue tracker at the main repository.
While It's true that we can't really edit DBC files because we would lose all the progress when re-extracted or lost the files, however, we can modify spells in a C++ file called SpellMgr.
There we have a function called SpellMgr::LoadDbcDataCorrections().
The main problem while doing this change is that we have to modify the core to support this change, and the function above contains a lot of corrections. Would need intense testing to make sure nothing is screwed up in the process.
In here by altering bits you can remove or add certain properties to the desired spells instead of touching the hard coded dbc files.
If you want an example, in this link, I have changed an Archimonde spell to have no cast time.
NOTE:
In this line, the commentary about damage can be miss leading but that's because I made a mistake and I haven't finished this pull request yet as of 18/04/2019.
The work has been started, notably by Kaev. I think at least 3 DBCs are now useless server side (but probably still needed client side, they are called DataBaseClient for a reason) like item.dbc.
Also, the original philosophy (for ALL cores, not just AC) was that we would not touch DBC because we don't do custom modifications, so there was no interest in having them server side.
But we wanted to change this and started to make them available directly in the DB, if you wish to help with that, it would be nice!
Why?
Because when emulation started, dbc fields were 90% unknown. So, developers created a parser for them that just required few code changes to support new fields as soon as their functionality was discovered.
Now that we've discovered 90% of required dbc fields and we've also created some great conversion tools for DBC<->SQL, it's just a matter of "effort".
SQL conversion is useful to avoid using of client data on server (you can totally overwrite them if you don't want to go against EULA) or just extends/customize them.
Here you are the issue about DBC->SQL conversion: https://github.com/azerothcore/azerothcore-wotlk/issues/584

Use Julia to perform computations on a webpage

I was wondering if it is possible to use Julia to perform computations on a webpage in an automated way.
For example suppose we have a 3x3 html form in which we input some numbers. These form a square matrix A, and we can find its eigenvalues in Julia pretty straightforward. I would like to use Julia to make the computation and then return the results.
In my understanding (which is limited in this direction) I guess the process should be something like:
collect the data entered in the form
send the data to a machine which has Julia installed
run the Julia code with the given data and store the result
send the result back to the webpage and show it.
Do you think something like this is possible? (I've seen some stuff using HttpServer which allows computation with the browser, but I'm not sure this is the right thing to use) If yes, which are the things which I need to look into? Do you have any examples of such implementations of web calculations?
If you are using or can use Node.js, you can use node-julia. It has some limitations, but should work fine for this.
Coincidentally, I was already mostly done with putting together an example that does this. A rough mockup is available here, which uses express to serve the pages and plotly to display results (among other node modules).
Another option would be to write the server itself in Julia using Mux.jl and skip server-side javascript entirely.
Yes, it can be done with HttpServer.jl
It's pretty simple - you make a small script that starts your HttpServer, which now listens to the designated port. Part of configuring the web server is that you define some handlers (functions) that are invoked when certain events take place in your app's life cycle (new request, error, etc).
Here's a very simple official example:
https://github.com/JuliaWeb/HttpServer.jl/blob/master/examples/fibonacci.jl
However, things can get complex fast:
you already need to perform 2 actions:
a. render your HTML page where you take the user input (by default)
b. render the response page as a consequence of receiving a POST request
you'll need to extract the data payload coming through the form. Data sent via GET is easy to reach, data sent via POST not so much.
if you expose this to users you need to setup some failsafe measures to respawn your server script - otherwise it might just crash and exit.
if you open your script to the world you must make sure that it's not vulnerable to attacks - you don't want to empower a hacker to execute random Julia code on your server or access your DB.
So for basic usage on a small case, yes, HttpServer.jl should be enough.
If however you expect a bigger project, you can give Genie a try (https://github.com/essenciary/Genie.jl). It's still work in progress but it handles most of the low level work allowing developers to focus on the specific app logic, rather than on the transport layer (Genie's author here, btw).
If you get stuck there's GitHub issues and a Gitter channel.
Try Escher.jl.
This enables you to build up the web page in Julia.

CSP Browser Policy + Zopim widget (+ underscore)

For the past few days, I've been trying to add properly Meteor's CSP package, browser-policy. So far, I followed these ressources:
https://dweldon.silvrback.com/browser-policy
https://themeteorchef.com/snippets/using-the-browser-policy-package/
Things were a bit rough at the beginning but we are close to something, the last piece of the puzzle being live-chat Zopim's widget not being a fan of our new policy. I tried to whitelist and put zopim's widget code into a Meteor.startup call somewhere but it still fails on load due to some unsafe-eval as you can see below.
As I don't want to loosen up more my policies, is there any workaround for this or should I just forget about Zopim and give a shot at some other tool (which I'd be glad to hear about if you have any suggestion).
Bonus: Also, I first had my policy with BrowserPolicy.content.disallowEval(); but MDG's underscore package started to fall appart and I had to allow it. Allowing eval is clearly not ideal and I'd be glad to hear any alternative.
Your're hitting the first bullet point from the "issues" section of my post. You have to decide if disallowing eval is more important to you than that particular 3rd party script. In our case, we allowed eval for a few days while the external script was modified (fortunately the creator agreed to the change). It never hurts to send an email and just explain that you think their scripts are posing a risk to your site because you can't enable a strict content security policy.
We currently have BrowserPolicy.content.disallowEval() set and haven't run into any issues. I find it hard to believe that a core package would violate that directive. Maybe some other package is causing it, but it's hard to say without a detailed analysis of your dependencies.

Convenient diff of patch files in Gerrit

My team using Gerrit for reviewing of changes and sometimes we have to push .patch for some files. Sometimes these .patches could reach over ~1000 lines (which obviously not good for reviewing). It is very inconvenient to review it as diff between patches itself. It would be better(in some cases) to review it as diff of origin file and origin file with applied patch(not diff between .patches). Even if original file isn't under version control, committer could attach it with patch set, right?
Unfortunately, after lasting googling i didn't find anything...
Is there any way to show diff between two patch files (not patch sets) in such approach or similar?
I think the best you can do is to find a three-way merge tool to compare
the original source;
the source with the old patch; and
the source with the new patch.
To use such a tool, I think you'd need to use the command lines the Gerrit interface provides to fetch the changes locally, then apply the patches, then use the merge tool for the review.
I don't think you'll find a way to get a "three-way merge" view of the code in the Gerrit UI itself.

Recommended method to download tweets based on search terms and store

I would like to download tweets based on certain search terms. I'm aware of HTTP GET and such techniques, but I'm not sure the best way to create a simple executable that downloads the tweets and saves them for subsequent analysis.
Any ideas? I'm a basic programmer - if you say "use curl" I know roughly what you mean but not how to set up an application to run curl commands!
Hence my dilemna.
Thanks in advance!
You absolutely can do it in c# or any other language.
From a very rudimentary standpoint, the Twitter API wiki will tell you how, but I know that's not what you're really asking.
I would suggest getting familiar with a good API such as Tweetsharp which also has methods not only for getting your typical timelines, but also using search. The advantage to this (aside from not having to handle your own serialization, etc.) is that it unifies the timeline and search calls as they are actually slightly different API's.
The downside to this approach though is that you're not going to be able to directly translate it to a mac, unless you write it using Silverlight.
the upside to this approach is that Tweetsharp gives you a number of options on how it gives you the data, which in turn gives you a number of options as to how to save the data.

Resources