i have a Hbase installation in distributed mode. The database is working fine and I am able to connect to the database if my webapp(spring + datanucleus JDO) is deployed on the same machine as the Hbase master.
But if I run the same webapp on a different machine I am not able to connect to the HBase server. There are no exceptions at all and the webapp just stalls and after a few minutes times out. My config files are as follows:
hbase-site.xml ->
<configuration>
<property>
<name>hbase.zookeeper.quorum</name>
<value><IP>:2181</value>
<description>The host and port that the HBase master runs at.
</description>
</property>
</configuration>
datanucleus.properties ->
javax.jdo.PersistenceManagerFactoryClass=org.datanucleus.api.jdo.JDOPersistenceManagerFactory
javax.jdo.option.ConnectionURL=hbase:
datanucleus.autoCreateSchema=true
datanucleus.autoCreateTables=true
datanucleus.autoCreateColumns=true
datanucleus.cache.level2.type=none
What am I doing wrong here. And also what all ports should be accessible to the client machines (as in production we open only required ports).
Thanks
The HBase is very sensitive to /etc/hosts configurations.. I had to set the zeekeeper bindings property in the hbase-site.xml correctly in order for the above mentioned Java code to work...For example: I had to set it as follows:
{property}
{name}hbase.zookeeper.quorum{/name}
{value}www.remoterg12.net{/value} {!-- this is the externally accessible domain --}
{/property}
{property}
{name}hbase.zookeeper.property.clientPort{/name}
{value}2181{/value} {!-- everything needs to be externally accessible --}
{/property}
{property}
{name}hbase.master.info.port{/name} {!-- http://www.remoterg12.net:60010/ --}
{value}60010{/value}
{/property}
{property}
{name}hbase.master.info.bindAddress{/name}
{value}www.remoterg12.net{/value} {!-- Use this to access the GUI console, --}
{/property}
The Remote GUI will give you a clear picture of the Binding Domains.. For example, the [HBase Master] property in the "GUI Web console" should be something like this: www.remoterg12.net:60010 (It should NOT be localhost:60010 )... AND YES!!, I did have to play around with the /etc/hosts just right as I didn't want to mess up the existing Apache configs :-)
One surprising thing that people sometimes don't think about: often most of a cluster is hidden behind a switch. This is often the case if all of your machine ip addresses are 198...*. If the machine you are trying to access the cluster from is not in the same network, you will not be able to use hbase. This is because your hbase connections are direct from the client to the node hosting the data. They do not go through the dual-homed node that you are accustomed to log into.
I don't know of a good work around, other than to put your application on the dual-homed node that can see the rest of the world. Or to set up some sort of gateway on that node. This may be a problem for libraries like JDO
Note, I'm not using DataNucleus. However, when we separated our Master from our client we had to adjust the configuration.
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", SERVER);
conf.set("hbase.zookeeper.property.clientPort", PORT);
You can look at your HBase web shell (usually: localhost:60010 on the master box) and get the ZK dump. Check if you're even making a connection to ZK.
Related
I am trying to import my local SQL server database into Azure and I have all the requirements (storage, bacpac file, etc). When I try to import the db, I am getting the error below.
The Azure SQL Server firewall did not allow the operation to connect.
To resolve this, please select the "Allow All Azure" checkbox in the
Sql Server's configuration blade.
I have already checked yes on the Allow Azure services and resources to access this server option in the firewall settings and added my client IP. Is there something behind the scenes preventing it from allowing access? I am running my SQL server on a Docker container.
Imported bacpac file
Import Operation Azure
Import Error
Firewall Settings
After a week of trial and error, the database imported fine with no problems so I'll answer my question. What is interesting is that I don't have a concrete answer to my question since I don't know exactly why it did, but I'll give two tips anyway.
It might have been the cache on Azure's side. I got in contact with
an Azure rep recently and they stated that the cache may have not
updated yet. Clearing your cache could be the source of the problem
as well. To clear the cache see this document.
DBCC FLUSHAUTHCACHE;
Creating a new rule that spans from 0.0.0.0 to 255.255.255.255 in
your firewall settings.
Feel free to provide more solutions in the answers. Like I said, it was likely the cache on their side. It was really odd that it didn't work for a while, even with the firewall settings configured correctly.
I want to set up a server (hosted on aws/or a running system in some part of the world) as an NTP server that can be queried globally.
Currently, I have modified the ntp.conf file on the node to be made the server as server . But the problem is, on using an NTP client if I try to query time from this server, or rather on using sudo ntpdate it says no suitable server found.
However, if I replicate the same on my local network (the server, as well as the querying node, are all on the local network) then this works perfectly fine.
I think the problem might lie in the ntp.conf file. Do I need to put some specific restrict lines for this to work publicly as well? And no I cannot list the server on public ntp pages. Is it at all possible?
Solved. This was a port issue. I was testing it on aws and had to manually open the related udp ports.
I have an ASP.NET 2 (very old) web app that I am trying to migrate to a new server. BTW, I am a complete ASP.NET noob but have a good amount experience with other web technologies.
In the web.config, I can see the following connection string XML:
<connectionStrings>
<add name="myCon"
connectionString="Data Source=DEVELOP\DEV_SQL2008R2;Initial Catalog=QO2_New;Persist Security Info=True;User ID=sa;Password=q"/>
</connectionStrings>
I did "nslookup" on the machine name 'DEVELOP' and RDP into the server. To my surprise, I don't see any SQL Server process running. Also, there is no SQL Server installed on that server. I am 100% that the app is running and I am very confident that there are content that comes from the database. I also did a search on the code base and it seems like the connection string is being used.
On a side note, we do have some other databases on separate machines. My questions are the following:
Can the machine DEVELOP somehow redirect the DB connection to somewhere else? If yes, how do I find out which IP is it being redirected at?
If DB connection cannot be redirected, then what is going here exactly?
Unfortunately, I have very little people whom I can ask what is going with this app. Any tips that you can share is much appreciated.
Nslookup uses DNS but there is a chance your machine has a hosts file entry that points DEVELOP to somewhere else..
However, if you RDP'ed into DEVELOP (as in, used that name when connecting RDP, not the ip address given by nslookup) then you should have been able to see a sqlserver install in the list of services.
If there is surely no sqlserver installed there then, from within your RDP session, use TCPView from sysinternals to see what process is listening on port 1433. Could be some kind of TCP redirector like PortTunnel from SteelBytes, bouncing the connection somewhere else. Remember to run tcpview on the server, not your local machine
Perhaps you can try getting connected to the sqlserver using management studio, using the credentials in the web config, you can get sqlserver itself to tell you more about where it is running, such as:
SQL - Query to get server's IP address
Using Environment variables in T-SQL
I am truly ashamed to inform everyone that the database in the web config was never being used. Therefore, even though it is pointing to a non-existing database, it still works. The data from the old database have been converted to static files.
I should have checked this the first thing I came across it. Thanks again for all the help.
I have an WCF Webservice project, built in my local machine, which when hosted using test client and triggered, returns values from remote database in JSON format.
For example, if you key in the URL with localhost then you get results back in the below format:
{"Id":3,"Value1":"67.5687","Value2":"126.7125"}
I want to host this project on a remote server with a public URL, which should return the above results back from any network. I have 3 question regarding this:
** What modifications should I do to my current WCF project to host it on remote server.
** Given the various types of hosting like :
1) windows process activation services (WAS)
2) IIS
3) Self hosting
4) Hosting in a Windows service,
which type of hosting is best suited for hosting on remote server.
** What changes should I make in my App.Config file (including the change in my endpoint address from localhost to IP address) to make the service work.
Thanks.
1) You shouldn't need to make any changes to your project just because you want to host the code on another machine. I find this an odd question.
2) Given your choice of JSON as data format and a browser as test client, I'm guessing you want to make it available over HTTP using simple GET requests. In the Microsoft stack, IIS is the web server, and the natural choice for this scenario.
3) It is quite impossible to answer. I don't know what's in your app.config today. I don't know if you're going to authenticate, and if so how. And I don't want to know! That said, it seems to me if everything is supposed to behave as it does on your dev box, the bindings are already ok. I don't remember if a WCF service needs to know about the endpoint it is itself at (hard to see why it would need to know this, really); I would have thought it more natural to do such configuration on the host, e.g. IIS. The client of course should use a different endpoint pointing to wherever you host the service. (You can put many endpoints in app.config and let the user choose one, btw.)
I think most of us sin against the following advice now and then, but it is the best advice I can give: Read a book. Learn as much as possible about the thing you're using, in this case WCF. You'll get the time back later, and your software will be less bad!
I'm thinking about configuring the remind calendar program so that I can use the same .reminders file from my Ubuntu box at home and from my Windows box at work. What I'm going to try to do is to make the directory on my home machine that contains the file externally visible through webdav on Apache. (Security doesn't really concern me, because my home firewall only forwards ssh, to hit port 80 my my home box, you need to use ssh tunneling.)
Now my understanding is that webdav was designed to arbitrate simultaneous access attempts. My question is whether this is compatible with direct file access from the host machine. That is, I understand that if I have two or more remote webdav clients trying to edit the same file, the webdav protocol is supposed to provide locking, so that only one client can have access, and hence the file will not be corrupted.
My question is whether these protections will also protect against local edits going through the filesystem, rather than through webdav. Should I mount the webdav directory, on the host machine, and direct all local edits through the webdav mount? Or is this unnecessary?
(In this case, with only me accessing the file, it's exceedingly unlikely that I'd get simultaneous edits, but I like to understand how systems are supposed to work ;)
If you're not accessing the files under the WebDAV protocol, you're not honoring locks set via LOCK and UNLOCK methods and therefore will open to potential to overwrite changes made by another client. This situation is described in the WebDAV RFC here: https://www.rfc-editor.org/rfc/rfc4918#section-7.2