Solr DataImportHandler - indexing multiple, related XML documents -

-

let's have 2 xml document types, , b, this:

a:

<xml>     <a>         <name>first number</name>         <num>1</num>     </a>     <a>         <name>second number</name>         <num>2</num>     </a> </xml>

b:

<xml>     <b>         <akey>1</akey>         <value>one</value>     </b>     <b>         <akey>2</akey>         <value>two</value>     </b> </xml>

i'd index this:

<doc>     <str name="name">first name</str>     <int name="num">1</int>     <str name="spoken">one</str> </doc> <doc>     <str name="name">second name</str>     <int name="num">2</int>     <str name="spoken">two</str> </doc>

so, in effect, i'm trying use value key in b. using dataimporthandler, i've used following data config definition:

<dataconfig>     <datasource type="filedatasource" encoding="utf-8" />     <document>         <entity name="document" transformer="logtransformer" loglevel="trace"             processor="filelistentityprocessor" basedir="/tmp/somedir"             filename="a.*.xml$" recursive="false" rootentity="false"             datasource="null">             <entity name="a"                 transformer="regextransformer,templatetransformer,logtransformer"                 loglevel="trace" processor="xpathentityprocessor" url="${document.fileabsolutepath}"                 stream="true" rootentity="true" foreach="/xml/a">                 <field column="name" xpath="/xml/a/name" />                 <field column="num" xpath="/xml/a/num" />                   <entity name="b" transformer="logtransformer"                     processor="xpathentityprocessor" url="/tmp/somedir/b.xml"                     stream="false" foreach="/xml/b" loglevel="trace">                     <field column="spoken" xpath="/xml/b/value[../akey=${a.num}]" />                 </entity>              </entity>         </entity>     </document> </dataconfig>

however, encounter 2 problems:

  1. i can't xpath expression predicate match rows; regardless of whether use alternative /xml/b[akey=${a.num}]/value, or hardcoded value akey.
  2. even when remove predicate, parser goes through b file once every row in a, inefficient.

my question is: how, in light of problems listed above, index data correctly , efficiently dataimporthandler?

i'm using solr 3.6.2 .

note: bit similar this question, deals 2 xml document types instead of rdbms , xml document.

i have bad experiences using dataimporthandler kind of data. simple python script merge data smaller current configuration , more readable. depending on requirements , data size, create temporary xml file or directly pipe results solr. if have use dataimporthandler, use urldatasource , setup minimal server generates xml. obvioulsy i'm python fan, it's quite it's easy job in ruby, perl, ...


Comments

Post a Comment

Popular posts from this blog

java - Jmockit String final length method mocking Issue -

-
using jmockit 1.2 jar, trying mock string's length method getting unexpected invocation exception: failed: test java.lang.illegalstateexception: missing invocation mocked type @ point; please make sure such invocations appear after declaration of suitable mock field or parameter @ stringdemo.testa$1.<init>(testa.java:17) @ stringdemo.testa.test(testa.java:13) i using testng: @test public void test() throws exception { new expectations() { @mocked("length") string astring; { astring.length(); result = 2; } }; system.out.println(a.showa("test")); } } actual class a: public class { public static int showa(string str){ int a= str.length(); return a; } } this wrong way of recording expected results. shouldn't mock , record string's length() method, record showa() instead. here solution @suppresswarnings("unused...
Read more

What is the difference between data design and data model(ERD) -

-
i have google question , find answer in day. know, data design many many table have list column,and mark pk , fk. why er-diagram have find this? data design process of designing database. main output of data design detailed logical data model of database. some people data design includes of needed logical , physical design choices , physical storage parameters needed generate design in data definition language. technically, database analysis, database analysts (dbas) trained do. while person can both data design , database analysis, these 2 different tasks. data design models data. database analysis takes model , applies 1 or more database engines (relational, hierarchical, nosql). a logical data model 1 of main outputs of data design. data model represented entity relationship diagram, or er diagram.
Read more

ios - Can NSManagedObject conform to NSCoding -

-
i need transfer single object across device. right converting nsmanagedobject dictionary , archiving , sending nsdata. upon receiving unarchiving it. transfer nsmanagedobject archiving , unarchiving instead of creating intermediate data object. @interface test : nsmanagedobject<nscoding> @property (nonatomic, retain) nsstring * title; @end @implementation test @dynamic title; - (id)initwithcoder:(nscoder *)coder { self = [super init]; if (self) { self.title = [coder decodeobjectforkey:@"title"]; //<crash } return self; } - (void)encodewithcoder:(nscoder *)coder { [coder encodeobject:self.title forkey:@"title"]; } @end nsdata *archivedobjects = [nskeyedarchiver archiveddatawithrootobject:testobj]; nsdata *objectsdata = archivedobjects; if ([objectsdata length] > 0) { nsarray *objects = [nskeyedunarchiver unarchiveobjectwithdata:objectsdata]; } the problem above code is. crashes @ self.title in initwithcoder sa...
Read more