I'm trying to setup a multi-NIC cluster with Cloudera5. Each node has an ethernet interface (eth1 - 172.17.2.x) plus an infiniband interface (ib0 - 192.168.69.x).
The problem is, the cluster communicates the infiniband addresses to the "outside world" when using HDFS.
I found out that the right parameter to get such a configuration working is "dfs.datanode.dns.interface" and that it has to be set to "eth1".
However, this parameter is not present in the Cloudera Manager interface. As it automatically overwrites the hdfs-site.xml file, I can't go to write it in the file.
I tried to use the Cloudera manager "Safety Valves" (Configuration > Service-Wide > Advanced > HDFS Service Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml), and set it to
<property>
<name>dfs.datanode.dns.interface</name>
<value>eth1</value>
</property>
but the HDFS Canary fails.
Could anyone please
Confirm that it's the right parameter
give me some help on how to set
it in the Cloudera Manager interface ?
Thanks in advance.
You can add further configuration properties that is not present in the CM interface filling these field in the HDFS configuration page:
- HDFS Service Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml"
- Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml
- HDFS Service Advanced Configuration Snippet (Safety Valve) for hadoop-policy.xml
In yout case you have to insert this code:
<property>
<name>dfs.datanode.dns.interface</name>
<value>eth1</value>
</property>
in the HDFS Service Advanced Configuration Snippet
Related
i would like cloud-init to not use 50-cloud-init.yaml. I have prepared my own file.
Do you know how to do this
You can add
network:
config: disabled
to /etc/cloud/cloud.cfg or a file in /etc/cloud/cloud.cfg.d.
Another option is to add
network-config=disabled
to the kernel command line.
While the network config yaml technically works as userdata, the network configuration will have already been written out before userdata is read.
One other option is to write your netplan configuration into /etc/netplan/99-some-name.yaml. If you have configuration there that overlaps with what is in 50-cloud-init.yaml, your configuration will override what is in the default configuration.
See https://cloudinit.readthedocs.io/en/latest/topics/network-config.html#disabling-network-configuration .
I'm attempting to run Spring Cloud Configuration Server, working through the examples in a book (Manning's Spring Microservices in Action), but updating to the latest versions: Java 17, spring-boot-starter-parent 2.6.1, with Spring Cloud 2021.0.0-RC1.
Each time I try to start the server, I get this error:
***************************
APPLICATION FAILED TO START
***************************
Description:
Invalid config server configuration.
Action:
If you are using the git profile, you need to set a Git URI in your configuration. If you have set spring.cloud.config.server.bootstrap=true, you need to use a composite configuration.
I am not using the git profile. I have tried two different profiles: native (with config files on the classpath) and vault (with a Hashicorp Vault server running locally). My latest /src/resources/bootstrap.yml contains the following:
spring:
application:
name: config-server
profiles:
active: vault
cloud:
config:
server:
vault:
port: 8200
host: 127.0.0.1
kvVersion: 2
server:
port: 8071
My best guess is that the bootstrap.yml file isn't getting picked up at server startup, and perhaps the git profile is a default. How can I remedy this?
OK, it looks like the problem here is that newer versions of Spring Cloud Configuration Server don't look for the bootstrap.yml file by default. There are a few different ways to solve it. The easiest is just to move all the properties to an application.yml/application.properties instead.
Another alternative is (found at NEWBEDEV here) is to include a dependency that implements the "legacy" bootstrap behavior:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>
add the following into application.properties file.
spring.application.name=techefx-spring-cloud-config-server
spring.cloud.config.server.git.uri=https://github.com/techefx/environment-variable-repo.git
server.port= ${port:8888}
Please go through the link below:
Spring Cloud Config File System Backend Issue (not reading properties from the file)
I'm trying to configure an extra interface in a wildfly swarm project.
The documentation says you just have to add the following snippets to the project-defaults.yml :
swarm:
network:
interfaces:
backnet:
bind: 192.168.4.5
But this doesn't work. Does anyone know if this is actually implemented?
I can add that setting the logging level to TRACE will output the generated xml configuration that shows only the default public interface and no backnet at all.
EDIT: I'm working with latest "stable" swarm : 2018.1.1
I've found a WA solution so far : add an XML snippets for interfaces:
I have to start the app like this:
java -Dswarm.local.ip=`hostname --ip-address` -jar myapp-swarm.jar -c config.xml
With this XML snippets (config.xml):
<?xml version='1.0' encoding='UTF-8'?>
<server xmlns="urn:jboss:domain:5.0">
<interfaces>
<interface name="mylocalif">
<inet-address value="${swarm.local.ip}"/>
</interface>
</interfaces>
</server>
This will be merged with the rest of the yaml generated conf and work. I still would like to do everything with YAML though.
Where is emrfs-site.xml on the EMR master node?
Consistent view is disabled within the EMR UI but I am unable to find the configuration file to verify.
sudo find / -name emrfs-site.xml
yields
/var/aws/emr/bigtop-deploy/puppet/modules/emrfs/templates/emrfs-site.xml
/usr/share/aws/emr/emrfs/conf/emrfs-site.xml
Neither of which seem to be relevant.
This is the relevant one:
/usr/share/aws/emr/emrfs/conf/emrfs-site.xml
It seems to be a strange place, but the path to the config file is hardcoded in java code. You can verify this by logging the debug output of the root logger to the console:
export HADOOP_ROOT_LOGGER="DEBUG,console
If consistent view is enabled, the following should be in that file:
<property>
<name>fs.s3.consistent</name>
<value>true</value>
</property>
I want to be able to check if my site is up and monitor it by nagios locally.
So I went to this site http://www.linickx.com/nagios which made the following, now while I cant see hosts.cfg so I created it and also services.cfg I had to create as well:
hosts.cfg
# host definition for bonus
define host{
use generic-host ; Name of host template to use
host_name bonus
alias bonus
address 92.12.35.162
check_command check-host-alive
max_check_attempts 10
notification_interval 120
notification_period 24x7
notification_options d,u,r}
services config
The following should appended to the end of hosts.cfg (located in /etc/nagios or /usr/local/nagios/etc)
# Service definition for bonus
define service{
use generic-service ; Name of service template to use
host_name bonus
service_description HTTP
is_volatile 0
check_period 24x7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups technical,support
notification_interval 120
notification_period 24x7
notification_options w,u,c,r
check_command check_http
}
How can I get this to work please?
Nagios won't add new configuration files unless either the file or directory is specified in the main nagios.cfg. Also, the new object configurations won't be added until you restart the Nagios process. Can you give some more detail as to how you installed Nagios (source or package) and what linux distro you're on?
Here are some additional resources:
Nagios Core Documentation
http://nagios.sourceforge.net/docs/3_0/
Nagios Core Support Forum
http://support.nagios.com/forum/viewforum.php?f=7