Getting the ExtractingRequestHandler to work in Solr - drupal

I am attempting to get Solr to work with Tika so I can index Word and PDF documents in my Drupal web site.
I've looked at the Wiki page and this page and they indicate adding a requestHandler in solrconfig.xml.
I did that and now Solr throws an exception:
org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.handler.extraction.ExtractingRequestHandler'
I have did some searches and see that others have had this problem but see no easy fix. I'm using Solr 3.4.0 on Windows Server 2003. Any ideas about how to resolve this?
As a side note I've got Drupal using Solr for searching and that is working. But what I cannot get working is to have Solr index PDF and Word documents. I'm sure this is a common need for most web sites but I have spent days on this and I cannot believe it is this poorly documented and this hard to figure out.

If you are running Solr from the example directory with the jetty setup, it should run as is without any changes.
However, for multicore setup you would need to copy the jars into the lib directory.
If you check the solrconfig in the example folders, it includes the jars for solr cell and extraction libraries.
solrconfig.xml -
Uncomment this line to include all the lib jars -
<lib dir="./lib" />
Copy the jars from these folders to your multicore lib folder.
These jars for used for extraction. (Apache pdfbox, poi, fontbox etc)
<lib dir="../../dist/" regex="apache-solr-cell-\d.*\.jar" />
<lib dir="../../contrib/extraction/lib" />
When you start Solr, you should see all the jars loaded.
Should get you working.

Related

How to use a license with JWrapper

So I have been working with the free version of JWrapper for some time now and have been quite pleased with the results; however, I have now purchased a license and would like to use it but I am unable to find the method with which to activate my JWrapper. I do not use the graphical interface version of JWrapper; rather I have created a xml installation file and pass that directly via commandline to JWrapper for building. Is there a special xml tag for specifying the license location? I was unable to find this information on JWrapper's homepage or support docs.
I received an answer from the support team. In order to use the license without using the JWrapperApp gui to build your application you will simply have to have the jwlicense.txt file in the same directory as your jwrapper.jar
After some tests it appears that the jwlicence.txt file must be in the current directory. So the best is to have all the files (jwrapper jar, jwrapper.xml and jwlicence.txt) in the same folder and run the compiler from that folder.

Using External jar causing Error

I'm doing the following steps:
Create an empty Notes Database (Lotus Notes 9.0.1 on Domino 9 Server)
Creating Xpage with only one Label
Inserting external Jar File (Code - Jars)
I tested this setting with different jar Files. In most cases there are no problems but I found 2 jar Files which produce an error 500 in XPage without being referenced
starface-rpc-1.6.442.jar
log4j-1.2.17.jar
Does someone has an idea what can be the problem?
Those two .jar files depend probably from other .jar files.
Look at the project pages like this for log4j to find out which .jar files you need to add to your application too.
Update:
Those .jar files stay in conflict with existing Java APIs used by Domino Server. That's why you get the error even without using them in an XPage as they get used by Domino Server executing an XPage.
Look here for explanation the log4j issue

Missing dependencies in latest rdfWebDeploy

I am trying to deploy a sparql endpoint using dotNetRDF to www.myapp.com/sparql.
rdfWebDeploy -test worked without errors but then running -deploy shows that there are missing assemblies in these tool.
First is complained about missing NewtonSoft.Json.Net35.dll, so I went into my nuget package and grabbed the dll in the net35 folder. However, this dll is named NewtsonSoft.Json.dll despite the 35 build...so I just renamed it has Net35 in the name and dropped it into rdfWebDeploy.
This worked so far, then it complained about MySql.Data.dll. I downloaded the MySql Connector which had the dll and dropped it into the rdfWebDeploy folder as well.
Finally rdfWebDeploy succeeded, however running the site produced
Could not load file or assembly 'Newtonsoft.Json.Net35' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference.
How can I get this tool to deploy to my Net 4.5 MVC site and get past this Newtonsoft issue? Also, how will I know my endpoint ("/sparql") is working, will it display something if I go there?
This tool hasn't been updated in a while and sounds like it has fallen behind the main code base somewhat, we haven't used the old .Net 3.5 builds of Json.Net in a long time nor do we have any dependencies on MySql.Data anymore. So this sounds like a bug which we'll look to fix in our latest release - this has been logged as TOOLS-424
In terms of what you will see when this is working it will depend on the configuration you have provided for your endpoint(s), a fairly default configuration would give you a query form such as the one seen on our live demo at http://www.dotnetrdf.org/demos/leviathan/ - you can see the corresponding configuration for our demos at http://www.dotnetrdf.org/demos/sampleConfig
Update
The mentioned bug has now been fixed and an up to date and working version of this tool will be available in the next dotNetRDF release currently scheduled for the end of July 2014.

Running Custom Deployer from Eclipse

I've gone through Jaime's deployer tutorial.
I've successfully created my deployer extension, which when integrating with SDL Tridion, the functionality works exactly as required.
But, what i can't get to work is the local debugging / running with the deployer inside eclipse (documented here)
The eclipse based deployer does run. If I drop my zip file into my test incoming folder the zip is picked up and processed. However, the customdeployer code I have written is never entered or executed.
I don't get any errors in the 'eclipse' deployer logs, but it always stops on the following line:
2012-04-13 20:24:51,642 DEBUG QueueLocationHandler - Removing exclusive lock on Deployment package: tcm:0-1026-66560 with type: CONTENT.
As we've three developers here also stuck on the same problem on all their machines I was wondering (hoping!) that this was a common problem and someone knew what we're doing wrong.
Thanks
Can you check which cd_deployer_conf.xml is it loaded by the Deployer? Just check the Deployer startup logs (in debug mode).
I suspect your Eclipse project at Debug/Run time doesn't load the the cd_*_config.xml files from the config folder in Eclipse. This will prevent your deployer module (which I supposed you configured in your cd_deployer_conf.xml) from being loaded and called.
What I normally do is to declare this config folder as an Eclipse Source Folder. Then at Debug/Run time, Eclipse will be included in the classpath automatically. This makes point #8 from http://www.sdltridionworld.com/articles/sdltridion2011/tutorials/Deployer_Extensions_With_Eclipse_3.aspx redundant.
I ran into exactly the same problem after following the same deployer extension tutorial.
I managed to solve it by changing the name of the package that my module was in to be com.tridion.deployer.extensions
Previously my module had been in a package I had named com.yourcompany.tridion.deployer.extensions and this appeared to have the affect of preventing the deployer from loading my extension module.
I had this issue, with a slight variation in that originally it worked, but then it stopped working.
Turns out the deployment package was somehow getting corrupted(locked?) in the process, as when i tried with a backup of the deployment package from the previous day it worked just fine.

How to make Qt aware of the QMYSQL driver

I'm trying to access a MySql database from a Qt application but I get the following error:
QSqlDatabase: QMYSQL driver not loaded
QSqlDatabase: available drivers: QSQLITE QSQLITE2
I find this very strange cause I have libqsqlmysql.so on my Qt folder. I have even tried to compile the MySql driver as a static plugin and add it to my .pro file as:
QTPLUGIN += qsqlmysql
But this also generates the same runtime error (it must've found the plugin cause there's no error compiling the application)
What am I missing? I would like to avoid having to compile Qt from source cause this will have to work seamlessly on the deploy machines as well.
BTW: Even though I'm developing and testing on Linux I will need to support Windows. Will I experience this same issue on Windows? How can I compile and link the MySql driver in both Linux and Windows?
The solution:
After following #Sergey's recommendations I did an strace of the application redirecting the output to grep so I could search for 'mysql' and for my surprise the application wasn't looking for the plugin at QTDIR/plugins/sqldrivers where I had libqsqlmysql.so, it was looking at QTDIR/lib. After copying the plugin to the lib folder the MySql connection worked.
Try opening the shared library with dlopen() and see if it loads and if not, what dlerror() tells you. I always run into similar problems on Windows. LoadLibrary()/GetLastError() saved me numerous times (last time it was because of a wrong version of some libiconv/libintl DLL). Running ldd on the plugin may also help.
If dlopen() works fine, try to load the plugin with QPluginLoader. If it doesn't load, then check the buildkey of the plugin. I usually do it the dirty way by running strings on the plugin and then looking for strings like "buildkey" or "QT_PLUGIN_VERIFICATION_DATA". Just looking at the build key and around it may give you an idea. For example, you may realize that you have compiled your plugin in the release mode while your application is compiled in the debug mode. In such case the build key won't match and the plugin won't load. Everything in the build key must match your configuration. Note that the version and the build key are checked differently: the build key must match exactly (or match some black magic called QT_BUILD_KEY_COMPAT), but in the version only the major version must match exactly, the minor version must be the version of Qt the plugin was compiled with or later and the patch level is ignored. So if your plugin was compiled with Qt 4.x.y then it will work with Qt versions 4.z.* where z>=x. This actually makes sense.
If the build key looks okay (which is unlikely if you got to this point), you may wish to look at QLibraryPrivate::isPlugin() source code to figure out what's wrong, but that doesn't look like an easy task to me (although running this in a debugger may help).
If QPluginLoader does load the plugin, check if it is in the right directory and has correct permissions. If you still didn't solve the problem by this point, it's time to look at the SQL module source code that actually loads these plugins. But it is extremely unlikely. I ran into this problem many, many times and it was always either the library not loading or the build key not matching.
Another way to go after QPluginLoader loads the plugin successfully is to use strace to figure out whether the program at least tries to open the plugin file. Searching for something like "sqldrivers" or "plugins" in the strace output should also give away the directory where Qt is searching for its plugins and specifically SQL drivers.
Update
Is it possible to compile the driver as a static plugin and don't worry about anything? Let's try:
d:\Qt4\src\plugins\sqldrivers\psql>qmake CONFIG+=static LIBS+=-Ld:/programs/Post
greSQL/lib INCLUDEPATH+=d:/programs/PostgreSQL/include
d:\Qt4\src\plugins\sqldrivers\psql>make
It compiles fine and now I got libqsqlpsql.a (release) and libqsqlpsqld.a (debug) in QTDIR/plugins/sqldrivers (it is the right place on Windows). I am using PostgreSQL driver here, but I don't think it will be any different for MySQL which I just don't have installed. Ok, let's compile some real program with it:
d:\alqualos\pr\archserv>qmake QTPLUGIN+=qsqlpsql PREFIX=d:/alqualos LIBS+=-Ld:/g
nu/lib INCLUDEPATH+=d:/gnu/include LIBS+=-Ld:/programs/PostgreSQL/lib LIBS+=-lpq
Note that I had to manually link to libpq, otherwise the linker would complain about undefined references. The funny thing is, qmake knows that qsqlpsql is located in QTDIR/plugins/sqldrivers and sets compiler and linker options accordingly. So it still needs to be in the right place to work, only you don't have to worry about your users running into the same problem as it is only used during compilation. An alternative would be to just use LIBS+=-Lpath/to/plugin LIBS+=-lqsqlpsql instead of QTPLUGIN+=qsqlpsql, at least the docs say that it should work, but I haven't tested it.
In order for the application to actually use the plugin I had to put the following in my main unit (CPP file):
#include <QtPlugin>
Q_IMPORT_PLUGIN(qsqlpsql)
It works! Also, from what I've been able to figure out from the sources, the build key and the version are checked only when a plugin is dynamically loaded (all the relevant stuff is in the QLibrary's private class, not even QPluginLoader's). So the resulting executable may (or may not, depending on the binary compatibility) work even with different versions and builds of Qt, although using it with older versions may trigger some bugs that were fixed later.
It is also worth noting that the order for loading SQL drivers is this: use the driver statically linked into Qt if available, then look for a driver registered manually with QSqlDatabase::registerSqlDriver(), then look for a driver statically imported into the application (the way described above), and finally try to load a shared plugin. So when you link statically, your users won't be able to use dynamically linked drivers they may already have, but will be able to use drivers linked statically into Qt (like in Ubuntu).
I compiled QT first and then realised that I need mysql as well. So I compiled mysql plugin by
executing following command in QT-DIR\src\plugins\sqldrivers\mysql folder.
Mysql plugin compile command
qmake "INCLUDEPATH+=$$quote(C:\Program Files\MySQL\MySQL Server 5.5\include)" "LIBS+=$$quote(C:\Program Files\MySQL\MySQL Server 5.5\lib\libmysql.lib)" mysql.pro
Plugings are then created in created in folder QT-DIR\plugins\sqldrivers.
However, when I tried to use it in my code. It failed with following error.
Error msg
QSqlDatabase: QMYSQLDriver driver not loaded
Solution
After some googling and checking Path variable I realised that the Mysql server lib
( C:\Program Files\MySQL\MySQL Server 5.5\lib) directory was not in my Path variable. I expect that the dll in this folder are used by the plugin at runtime. After including Mysql server lib in Path variable everything worked smoothly. Hope this information saves some hair on other programmers scalp, as I uprooted quite a few. :D
Last time I looked at this you needed to rebuild Qt from source and include the appropriate MySQL source.
Building Qt from the sources is not hard, it just takes a while. You are likely to have the required tools already.
A possible workaround may be to access the back-end over ODBC instead.
In order for your app to pick up the plugin at runtime, the shared library implementing the MySQL plugin needs to be placed in the correct directory. The best way of determining that directory is to check the output of QCoreApplication::libraryPaths. You can also force specific paths by using a qt.conf file.
Please note that plugins must be placed in subdirectories within the plugin path, and the final part of the path name (i.e., the parent directory of the shared libraries) cannot be changed. SQL drivers need to go in a directory named sqldrivers, i.e. <pluginpath>/sqldrivers. For more details on plugin directories, see How to Create Qt Plugins.
I was experiencing this same issue as well. I've been installing and experimenting with a lot of different Python tools and UIs. I then uninstalled everything python related. I did a fresh install of Python 3.2, PyQT 3.2, and Eric5. No more errors with the QMySQL driver.
well i have had this issue, and after a lot of time, and different tools, i found that QT ( on windows, have not been able to test on Linux.) loads the "QSQLMYSQL.." when requested, but before runtime the lib ("QSQLMYSQL..") file must reside on one of the searched paths (QApp.libraryPaths()) inside a folder called "sqldrivers".. otherwise QT will just ignore the file, even if it is at some other point inside the searched path.
what i did was to monitor the dependency of a sample app, and when i removed the "QSQLMYSQL.." dll from "plugins\sqldrivers\" it failed, but when i maded a folder inside the app folder, called "sqldrivers" and placed the "QSQLMYSQL..." inside there, it loaded.
what i have is mysql 5.5, qt 4.7.4.
hope anyone can use this, and if anyone knows more about it, i would like to know where to find it(http://doc.qt.nokia.com/stable/sql-driver.html, is the closest you can get to the information about the folder structur). :P
This may also happen if your QMYSQL plugin is linked against the "wrong" mysql_client.a or it isn't in the LD_LIBRARY_PATH. I had this problem on OSX because mysql was installed via ports, and I fixed it with:
install_name_tool -change libmysqlclient.18.dylib /usr/local/mysql/lib/libmysqlclient_r.18.dylib libqsqlmysql.dylib

Resources