I built a reverse proxy with Apache 2.4 on a cento's 7 server. It works with standard html pages but i need to substitute some url stored in .js files too. The directive:
ProxyHTMLExtended On
should enable the parsing inside external .css and .js files but it doesn't work. In the log file I can see:
[proxy_html:trace1] [pid 3263] mod_proxy_html.c(823): [client xxx] Non-HTML content; not inserting proxy-html filter
I tried to use mod_substitute, this is the interesting part in my httpd.conf:
ProxyPass /mylocation/ http://remoteserver/
<Location /mylocation/>
ProxyHTMLEnable On
ProxyHTMLExtended On
LogLevel debug proxy_html:trace3 substitute_module:debug
RequestHeader unset Accept-Encoding
AddOutputFilterByType SUBSTITUTE text/javascript text/html
Substitute "s|/css/|/mylocation/css/|ni"
Substitute "s|/js/|/mylocation/js/|ni"
Substitute "s|/custom_logo/|/mylocation/custom_logo/|ni"
Substitute "s|/html/|/mylocation/html/|ni"
Substitute "s|/current_config/|/mylocation/current_config/|ni"
Substitute "s|/web_lang/|/mylocation/web_lang/|ni"
Substitute "s|/custom_lang/|/mylocation/custom_lang/|ni"
ProxyPassReverse /
ProxyHTMLURLMap //remoteserver /mylocation/
ProxyHTMLURLMap http://remoteserver /mylocation/
ProxyHTMLURLMap /mylocation /mylocation
ProxyHTMLURLMap ^\/(.*) /mylocation/$1 R
</Location>
But in the log file there aren't any mod_substitute trace. It seems mod_substitute is never called.
The proxyHTMLURLMap rules works fine, but only on regular html files.
Depending on the .js file I'm asking to the server, I can see in the log file:
[xml2enc:debug] [pid 3259] mod_xml2enc.c(254): [client xxx] AH01434: Charset ISO-8859-1 not supported by libxml2; trying apr_xlate
or
[proxy_html:trace1] [pid 3263] mod_proxy_html.c(823): [client xxx] Non-HTML content; not inserting proxy-html filter
then the process stop, I receive the file but nothing has been replaced on it.
1) Wy the "ProxyHTMLExtended On" rule don't parse external .js files as described in Apache documentation?
2) Wy the mod_substitute don't work?
I'll try to answer to your questions
1) Wy the "ProxyHTMLExtended On" rule don't parse external .js files as described in Apache documentation?
You say that ProxyHTMLExtended Directive:
should enable the parsing inside external .css and .js files but it doesn't work.
That's seems to be incorrect, the current doc says:
Set to Off, HTML links are rewritten according to the ProxyHTMLURLMap directives, but links appearing in Javascript and CSS are ignored.
Set to On, all scripting events (as determined by ProxyHTMLEvents) and embedded scripts or stylesheets are also processed by the ProxyHTMLURLMap rules, according to the flags set for each rule. Since this requires more parsing, performance will be best if you only enable it when strictly necessary.
That means that embedded scripts, the ones in <script></script> are checked. It doesn't mention .js files.
2) Wy the mod_substitute don't work?
About this i don't know for sure why it doesn't work, but assuming the mod_substitute is enabled since apache started without error, the only thing that I can guess is that apache is sending application/javascript as Mime-Type instead of text/javascript that you wrote
Some bonus suggestions:
I wouldn't use ProxyHTMLURLMap ^\/(.*) /mylocation/$1 R with ProxyHTMLExtended On because will translate every / in your scripts, if you have <script> var a = 12/2; </script> will be translated into <script> var a = 12/mylocation/2; </script>. I would consider using ProxyHTMLURLMap / /mylocation/ c (the c flag means: "Pass embedded script and style sections through untouched.")
I don't really think that you need ProxyHTMLURLMap /mylocation /mylocation
ProxyHTMLURLMap http://remoteserver /mylocation/ will add an extra / to your urls, it still works but, imho, it is not a good translation.
Ex. <a href="http://remoteserver/about"> becomes <a href="/mylocation//about">
I suggest to rewrite it like this ProxyHTMLURLMap http://remoteserver /mylocation
I have a similar problem where mod substitute was not working for me.
Then I read somewhere that normally, mod_subsittue is actually by default only working if the HTTP response you get from the server has mime type txt/html.
This was not my scenario.
My scenario was that I want to re-write the content of an XML, namely a JEE web service that was being reverse proxied by apache httpd.
In order to manage to make the mod substitute alter the reply content it was necessary to do:
<Location /mockOutgoingWebServicePortBinding>
# core authentication and mod_auth_basic configuration
# for mod_authn_dbd
AuthType Basic
Authname "Password Required"
# core authorization configuration
AuthUserFile C:\Apache24\conf\htpasswd
Require valid-user
# mod_filter to be able to subsitute text xml
AddOutputFilterByType SUBSTITUTE text/xml text/html
Substitute "s|http://someHostName:8088/|http://localhost:80/|i"
</Location>
The magic step was enabling the mod_filter and adding the directive: AddOutputFilterByType .
When this was added in, the substitue altered the body of the xml. replacing the endpoint address.
About this:
But in the log file there aren't any mod_substitute trace
I managed to get a mod_substitute trace in error.log by using:
LogLevel alert substitute:trace8
Related
We had a security check on our application, and it reported a few issues like missing charset attribute for .css and .js files.
Attack Type: Undefined charset attribute
Error Description: The Content-Type HTTP header is missing charset attribute
Content-Type: text/CSS
I found this answer The Content-Type HTTP header is missing charset attribute
Our application has been developed using play framework and I tried to add in the application.conf like this
AddDefaultCharset : utf-8
AddCharset utf-8 = [.htm .html .js .css]
But it didn't work. How can I resolve this issue in our application?
Thanks
Maybe You can try this one.
application.conf
play.filters.enabled += "play.filters.headers.SecurityHeadersFilter"
change this one, as Play Docs instructed.
play.filters.headers.contentTypeOptions -
sets X-Content-Type-Options, “nosniff” by default.
Source here.
System details:
Server: Apache, Ubuntu 16.02
Client: Windows 10, Chrome
My apache config file looks as follows:
<VirtualHost *:80>
ServerName mycoolsiteA.mydomain.com
ServerAlias mycoolsiteB.mydomain.com
Alias /static /var/www/python/mysite/static
Alias /templates /var/www/python/mysite/templates
WSGIDaemonProcess my_app user=www-data group=www-data threads=5
WSGIScriptAlias / /var/www/python/mysite/start.py
WSGIScriptReloading On
<Directory /var/www/python/mysite>
WSGIProcessGroup my_app
WSGIApplicationGroup %{GLOBAL}
order deny,allow
Allow from all
</Directory>
</VirtualHost>
When I load the css in a browser it is showing the following. However, this is quite random. It "sometimes" works, and sometimes does not.
My example url is this:
https://mycoolsiteA.mydomain.com/static/css/bs/bootstrap.min.css
We do have a Firewall appliance which auto-redirects http to https, but I'm loading the css directly with https.
And the result looks like this:
Yet, if I refresh several times, or add a r=number on the end, it will clear the cache and load. But, it will do it again randomly until I clear the cache again.
Here it is loaded and working:
Any thoughts, direction on where to look to fix this?
Welcome to the fun world of UTF-8!
I've seen this dozens of times in PHP (see UTF-8 all the way through). What you're experiencing is almost certainly a UTF-8 file being delivered as some other character encoding (or vice-versa). I downloaded Bootstrap and Notepad++ identifies them as ANSI encoded. Your web server might be forcing that to use UTF-8. Look at the response headers for something like this
Content-Type: text/css; charset=UTF-8
So your web server is saying it's UTF-8 but it's really not. At which point your browser starts vomiting out �, meaning it doesn't recognize the character.
There's a couple of ways to deal with this if that's the case.
Re-encode the file as UTF-8 (Notepad++ can do this for you)
Stop passing a default UTF-8. Your browser will try to identify the encoding
Just for kicks, try loading the CSS directly from the MaxCDN servers
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
They do not pass a UTF-8 header
I spent days now in researching on how to add some headers to nginx. All I try to do is adding these lines:
location ~ ^/(assets)/ {
add_header Access-Control-Allow-Origin *;
}
What is the best way to put these lines into the nginx.conf?
Is there also a way to not overwrite the standard nginx.conf just in case beanstalk updates the settings so I wont miss it?
The default elastic beanstalk nginx.conf seems to have this line toward the end :
include /etc/nginx/conf.d/*.conf;
(Well, I can tell you that's what the file looks like for the docker solution stack versions 1.4.1 and 2.0.4, no idea if that's guaranteed across all solution stacks).
So I think one way would be to to drop a file named whatever.conf into the /etc/nginx/conf directory using the ebextensions mechanism .
I have been look around trying to get this working right for a while now, and finally bit the bullet and posted here.
I've got a LAMP stack with ModSecurity using the OWASP core rule set (v 2.2.5) and just installed Wordpress. I expected conflicts with ModSecurity, but I haven't been able to ignore or work aroudn the only error I've encountered so far.
The Apache error.log file and the modsec_audit.log both list the same error:
ModSecurity: Rule 7f5d9a449228 [id "950901"][file "/etc/modsecurity/owasp-crs/activated_rules/modsecurity_crs_41_sql_injection_attacks.conf"][line "77"] - Execution error - PCRE limits exceeded (-8): (null).`
I've tried creating a new .conf file where the crs conf files are located containing
<LocationMatch .*>
<IfModule mod_security2.c>
SecRuleRemoveById 950901
</IfModule>
</LocationMatch>
and even removed the IfModule statement and then LocationMatch statement when it didn't work.
Finally I resorted to commenting out lines 76 and 77 in the .conf file, and the error still appeared. This also had no effect.
Only changing SecRuleEngine to Off in modsecurity.conf finally let me access the page. Of course this defeats the purpose of ModSec.
Where am I going wrong?
Try adding this to your php.ini file (or included conf file):
pcre.backtrack_limit = 10000000
pcre.recursion_limit = 10000000
And then this to your modsecurity.conf:
SecPcreMatchLimit 150000
SecPcreMatchLimitRecursion 150000
This should allow for recursion without having to fully disable mod_security.
I have a gzip-compressed style and scripts in files:
/scripts.js.gz
/styles.css.gz
The problem is that it does not serve the correct mime-type and browser does not recognize, that the files are compressed css or js (browser recognize the type as application/octet-stream, where it should be text/css or so).
I tried adding the following to mime.type of nginx, but with no effect. I suppose, it does not recognize, that the file is compressed.
types {
text/css css.gz;
application/x-javascript js.gz;
}
When trying to access the files, the browser handle the files as files to download and not to present.
I had the same problem, coming from Apache.
The problem I found is, the types block does not allow you to specify .css.gz as a file extension. However, the location block does! My solution is to make a location for .css.gz and then modify the content type for .gz within that location, like this:
location ~ \.css\.gz$ {
add_header Content-Encoding gzip;
gzip off;
types {
text/css gz;
}
}
Do not change the mime types and use the Gzip Static Module to handle this.
When this is active, Nginx will try to serve "file.ext.gz" first and then try "file.ext", if not found, for all requests within the context (location etc) where it is active.