Commit Graph

339 Commits

Author SHA1 Message Date
David Sauve 144c4e6225 Updated version number to 1.1.3beta 2010-01-27 09:18:42 -05:00
David Sauve 6596259b2d Fix for issue #31 -- OverFlow error. end_offset was being improperly calculated. 2010-01-27 08:46:37 -05:00
David Sauve 5f3ba78a26 Changed SearchResult.score to be the document percentage score instead of the weight. Added a new attribute that is the weight. 2010-01-19 15:27:16 -05:00
David Sauve 2864744bf3 Updated SearchQuery to accept an optional argument 2010-01-18 16:11:43 -05:00
David Sauve 5dcdaf1751 Fix for issue #36 -- DatabaseModifiedError 2010-01-15 10:02:04 -05:00
bkonkle 9ba7d0f8d5 Adjusted the query operations for boosts 2010-01-15 22:10:28 +08:00
David Sauve 4f0f219ede Updated with new version number 2009-12-09 11:29:34 -05:00
David Sauve 7626ee6e38 Moved schema generation out of database opening method and made it into a property that should be cached. 2009-12-08 13:18:17 -05:00
David Sauve d2750c58fa Added a friendlier error for when the index can not be opened 2009-12-08 12:21:04 -05:00
David Sauve f592114415 Added InvalidIndexError and test for write permissions on path 2009-12-08 11:44:10 -05:00
David Sauve 925b74c72c Fixed __version__. Had incorrect version number. 2009-12-08 09:58:23 -05:00
David Sauve 858fcf1523 Added a friendlier error for when the index can not be opened 2009-12-08 09:56:57 -05:00
David Sauve bd319a8a3a Removed a TODO that doesn't apply anymore 2009-12-08 09:19:06 -05:00
David Sauve 7d06ac5c89 Some docstring fixes 2009-12-07 20:02:39 -05:00
David Sauve f560d9eefe Updated release number 2009-12-07 09:57:03 -05:00
David Sauve 1ec4fdf7ab Added stemming support 2009-12-05 11:32:29 -05:00
David Sauve 72134e096b Query facets working again and reworked narrow to take a query_string instead of xapian.Query. This should be easier to use. 2009-12-05 10:43:52 -05:00
David Sauve 3752844e4a Fix for date facet when gap by month amount is larger than 1 2009-12-04 20:45:36 -05:00
David Sauve f1b7c04c12 Only thing missing now is query_facets 2009-12-04 20:30:26 -05:00
David Sauve 1fe78fc384 __lt is working 2009-12-04 16:44:36 -05:00
David Sauve 999800e4a8 __gt is working 2009-12-04 16:41:41 -05:00
David Sauve 253382b41a __startswith has been implemented in new branch 2009-12-04 15:41:32 -05:00
David Sauve 4ea6f5eda1 Added parse_query utility method for SearchBackend. This takes a query_string and attempts to convert it to a xapian.Query for use by search 2009-12-04 14:42:06 -05:00
David Sauve da34c32849 Added LiveLiveXapianSearchQueryTestCase 2009-12-04 14:16:08 -05:00
David Sauve 617f623b6b Order by is working 2009-12-04 10:27:23 -05:00
David Sauve 012ba98ed1 Added limit_to_registered_models in mlt 2009-12-03 16:48:21 -05:00
David Sauve 585ccfdac7 More like this is working 2009-12-03 16:40:33 -05:00
David Sauve ccde83ff8c Highlighting is working again 2009-12-03 13:49:26 -05:00
David Sauve a6e7670973 Added spelling suggestion 2009-12-03 12:20:04 -05:00
David Sauve 60e8925280 Changed is not to is when testing for list, tuple. Silly style thing. 2009-12-03 10:21:03 -05:00
David Sauve 4bff8c9376 Split marshalling into two methods: marshal_term and marshal_value as they should be done differently 2009-12-03 09:38:49 -05:00
David Sauve c2e33da295 Removed a bunch of search tests that were testing various query_string combinations. These are no longer relevant. 2009-12-02 15:05:53 -05:00
David Sauve 6ffeb96090 Removed SearchBackend.delete_index. Is not part of the API. 2009-12-02 13:35:00 -05:00
David Sauve 8351f79326 SearchBackend.update is working again 2009-12-02 13:10:30 -05:00
David Sauve 4b0f6520f1 Re-added xapian_backend tests. This is in a really rough state right now. Most tests are commented out. 2009-12-02 11:47:26 -05:00
David Sauve c6dd7b5deb Updated to GPLv3 to maintain compatibility with Apache license 2009-12-01 15:55:22 -05:00
David Sauve 2dd8104abf Added some docstrings to private methods that should make their uses more apparent 2009-12-01 09:19:30 -05:00
David Sauve d197014a82 Added support for not in 'in' based queries 2009-12-01 09:11:01 -05:00
David Sauve 09c12d88f7 Refactored some of the comon bits of query generation for phrase, all, and term based queries. Fixed 'in' based queries 2009-12-01 08:58:34 -05:00
David Sauve f027e26644 More refactor to keep things DRY 2009-11-30 16:49:47 -05:00
David Sauve bbb38e3ef6 Small refactor to make even 'exact' more DRY 2009-11-30 16:36:29 -05:00
David Sauve 2065ecee7f Small refactor to make 'exact'more DRY 2009-11-30 16:34:40 -05:00
David Sauve 99de351c79 Some minor formatting tweaks to code 2009-11-30 16:20:05 -05:00
David Sauve 2e3ed62c02 A whole lot of work on phrase and not operators. 2009-11-30 16:10:46 -05:00
David Sauve 12a7008777 Added a comment explaining the pure boolean subquery line 2009-11-30 06:26:34 -05:00
David Sauve 184f10af2d Converted model filtering to use pure boolean subquery 2009-11-29 19:12:59 -05:00
David Sauve 2a4b32f03b SearchQuery now supports model filtering 2009-11-29 17:06:29 -05:00
David Sauve 7dd36f47d8 Removed RESERVED_WORDS and RESERVED_CHARACTERS 2009-11-29 16:31:13 -05:00
David Sauve 3cacb54cf5 Added term boosting 2009-11-29 16:05:36 -05:00
David Sauve e9f2064454 Added tests for boolean query and datetime query. Also added test for phrase query. 2009-11-28 18:55:11 -05:00
David Sauve 1ce5b2ca14 Merged more master changes into next 2009-11-28 12:34:21 -05:00
David Sauve 500664a2e4 Merged master changes into next 2009-11-28 12:31:03 -05:00
David Sauve bff2a62f93 Small cleanup of unused cruft in SearchQuery 2009-11-19 15:06:12 -05:00
David Sauve b94eee10c5 Remove unused build_sub_query method 2009-11-13 11:48:16 -05:00
David Sauve 537b1802a3 Five tests. OR operator now working 2009-11-13 10:01:16 -05:00
David Sauve 09ffc6d481 Four tests passing now. Recursively parsing the search nodes and negated on NOT as required. 2009-11-13 10:01:15 -05:00
David Sauve 35f51e97bc Passing three tests. Empty query, single content value, multi-content values 2009-11-13 10:01:15 -05:00
David Sauve 529edc24b4 Passing first two tests... 2009-11-13 10:01:15 -05:00
David Sauve 30bc0d0805 More refactor work 2009-11-13 10:01:15 -05:00
David Sauve c89addc959 Work on refactoring. Eliminated a lot of useless code and started to implement build_query using xapian.Query 2009-11-13 10:01:15 -05:00
David Sauve 8c61cdc9c2 More changes to build_query 2009-11-13 10:01:14 -05:00
David Sauve 3334b94349 Started work in refactor 2009-11-13 10:01:14 -05:00
David Sauve d7e606f0d5 Five tests. OR operator now working 2009-11-13 10:00:37 -05:00
David Sauve 3c42660a47 Four tests passing now. Recursively parsing the search nodes and negated on NOT as required. 2009-11-13 10:00:37 -05:00
David Sauve 3280b89d73 Passing three tests. Empty query, single content value, multi-content values 2009-11-13 10:00:37 -05:00
David Sauve ef70ade65a Passing first two tests... 2009-11-13 10:00:36 -05:00
David Sauve 9b3b6d0b02 More refactor work 2009-11-13 10:00:36 -05:00
David Sauve df46eea292 Work on refactoring. Eliminated a lot of useless code and started to implement build_query using xapian.Query 2009-11-13 10:00:36 -05:00
David Sauve 8cfde151dd More changes to build_query 2009-11-13 10:00:36 -05:00
David Sauve c7744fee70 Started work in refactor 2009-11-13 10:00:36 -05:00
David Sauve a96ed9e216 Five tests. OR operator now working 2009-11-13 09:59:00 -05:00
David Sauve 84988c69f3 Four tests passing now. Recursively parsing the search nodes and negated on NOT as required. 2009-11-13 09:58:59 -05:00
David Sauve 5905909b7c Passing three tests. Empty query, single content value, multi-content values 2009-11-13 09:58:59 -05:00
David Sauve ac11ba627b Passing first two tests... 2009-11-13 09:58:58 -05:00
David Sauve 96f16cb4e6 More refactor work 2009-11-13 09:58:58 -05:00
David Sauve 555b13a5de Work on refactoring. Eliminated a lot of useless code and started to implement build_query using xapian.Query 2009-11-13 09:58:58 -05:00
David Sauve 30a8f8fc0c More changes to build_query 2009-11-13 09:58:57 -05:00
David Sauve 44bb6c6f48 Started work in refactor 2009-11-13 09:58:57 -05:00
William Shallum ad6b86e1da use time.strptime for compatibility with python 2.4 2009-11-13 04:34:31 +08:00
David Sauve abd5920645 Updated to use changed get_identifier method 2009-11-09 20:27:46 -05:00
David Sauve 204feae63e Updated to use new SQ objects 2009-10-21 10:22:29 -04:00
David Sauve fbeb54fa20 Modified SearchBackend to accept limit_to_registered_models in order to properly support in SHA b329b9795 of Haystack 2009-10-08 14:42:58 -04:00
Jannis Leidel a7e40c7482 Removed uneeded __init__.py in root dir. Added __version__ and __author__ information to backend file. 2009-10-02 00:35:34 +08:00
David Sauve b1f5ba9f13 Removed outdated warning regarding query facets 2009-09-17 14:00:36 -04:00
David Sauve 520713612a Added support for spelling_query in run method as added in django-haystack SHA 167030ae1e34e8e4ab5f765b5c4fcd62e6c7ad69 2009-09-17 13:39:52 -04:00
David Sauve e0c6b3d7b1 Added support for faceting when using MultiValueField 2009-09-16 14:07:24 -04:00
David Sauve 33ed2ce8a9 Changed highligting to not highlight any reserved words 2009-09-01 19:44:23 -04:00
Joshua Jonah c1fc676ecf Added ignore of 'select' operators in highlighting code. Should be replaced with a proper stopwords.txt file that can be optionally added as a settings.py constant.
Signed-off-by: David Sauve <dnsauve@gmail.com>
2009-09-02 06:11:54 +08:00
David Sauve 4761286851 Added missing docstring argument on _query method 2009-09-01 16:14:34 -04:00
David Sauve d593924386 Added float type to build_schema. Floats were being indexed as text. 2009-08-29 18:04:28 -04:00
David Sauve 4ea8271fb3 Changed the way int/long values are stored so that they are not serialised as xapian types. Instead, use 0 padded strings. 2009-08-28 10:42:58 -04:00
David Sauve 7c00117043 Fixed query string generation to work with NOT, gte, lte, lt, gt, and in. 2009-08-27 20:20:02 -04:00
David Sauve 6922b05f47 Better exclude query generation 2009-08-27 17:47:14 -04:00
David Sauve d8b709a80c Revert "Reverted not ordering. Put back check to push not to the end of query string."
This reverts commit a7fe00c058.
2009-08-18 21:19:51 -04:00
David Sauve a7fe00c058 Reverted not ordering. Put back check to push not to the end of query string. 2009-08-18 21:06:12 -04:00
David Sauve 3fd41552c3 Removed requirement stipulating first expression cannot be NOT. 2009-08-18 20:26:59 -04:00
David Sauve 2c406c7ace Allow exclude to work with in__ operator 2009-08-18 18:57:04 -04:00
David Sauve c02e0ecc01 Small refactor to move flag generation back into it's own private method. This way, it can be used by query and narrow query generation with out repeating code. 2009-08-18 16:30:15 -04:00
David Sauve bb38e1b83e Enable wildcard support on demand instead of always. Should reduce memory usage a bit. 2009-08-18 16:16:41 -04:00
David Sauve c113c0face Fix to ensure multiple expression query strings that begin with NOT are parsed corectly. This also changes the default flags to only use FLAG_PURE_NOT when needed 2009-08-18 15:52:34 -04:00
David Sauve c7c6b21e20 Added custom ExpandDecider. To be used in 'more_like_this' calls 2009-08-18 09:49:20 -04:00
David Sauve 6d2f4a53a7 Small change to MSetIter use. Using built in members instead of method calls 2009-08-18 08:56:35 -04:00
David Sauve 9cf9fe9e27 Updated date_facet to deal with changes in SHA d1a195 in Haystack. 2009-08-16 14:03:19 -04:00
David Sauve 9754ba0338 Updated to include build_schema method to deal with changes introduced in Haystack SHA 4f3af88c. BACKWARDS INCOMPATIBLE. 2009-08-16 11:40:47 -04:00
David Sauve d31a875b46 Added support for additional_query_string in mlt 2009-08-14 19:06:23 -04:00
David Sauve 051fe7ddd4 Fixed bug in 'mlt'. Was using not allowing additional_query_string, start_offset, and end_offset params 2009-08-14 18:54:46 -04:00
David Sauve b42ac852d5 Better field facet implementation 2009-08-11 09:28:15 -04:00
David Sauve 648ca3a3d3 Added docstring for _do_query_facets 2009-08-11 08:49:45 -04:00
David Sauve 1a8e6d24b6 Implemented query facets 2009-08-11 08:42:42 -04:00
David Sauve 6a82cb5e3c More date facet tests 2009-08-10 16:38:13 -04:00
David Sauve 5be650c426 Minor fix to date based facet 2009-08-10 16:34:29 -04:00
David Sauve 54ae6c3694 Initial date based query implementation. Actually working. 2009-08-10 16:12:59 -04:00
David Sauve ea02d182ff Properly built date gaps. Still need to properly count 2009-08-08 08:00:54 -04:00
David Sauve e517d3a1e1 resolved merge conflict 2009-08-07 19:46:04 -04:00
David Sauve 25eede14d8 Still working on date-based facets 2009-08-07 19:41:24 -04:00
David Sauve d1715de84b More date facet updates. Not ready for merging by a long shot... 2009-08-07 16:47:49 -04:00
David Sauve 40863f7413 Roughed in the date_query facet. Not working yet. 2009-08-07 16:21:29 -04:00
David Sauve 1ef902e849 Implemented boost 2009-08-07 15:58:36 -04:00
David Sauve 8015dd8297 Roughed in the date_query facet. Not working yet. 2009-08-07 14:22:40 -04:00
David Sauve e90262b1e4 Added custom run and run_mlt methods to SearchQuery 2009-08-06 19:22:42 -04:00
David Sauve 69899cd215 Ascending and descinding sortin was inverted. Fixed. Also added a test to confirm float sorting is working as expected. 2009-08-06 09:03:03 -04:00
David Sauve a6a1859029 Minor tweak to cleanup. Using shutil instead of iterating file objects 2009-08-05 16:44:12 -04:00
David Sauve bc4f8a2be2 Better schema generation 2009-08-05 09:28:44 -04:00
David Sauve 1f84935b6b Minor adjustment to gt, gte, lt, lte query generation. gt was gte, lt was lte, and vice-versus. 2009-08-03 08:48:55 -04:00
David Sauve e621a731b3 Fix for issue #12 -- Index Field with index=False raises KeyError 2009-08-03 08:47:19 -04:00
David Sauve 068a657bc9 Fixed query generation to match new value range processor expectation 2009-08-02 20:33:47 -04:00
David Sauve e385129a6d Added support for gt, gte range queries 2009-08-02 20:32:19 -04:00
David Sauve 3841376be8 Added support for wildcard in lt, lte query 2009-08-02 20:14:51 -04:00
David Sauve 503d70693e Removed _value_type_column method 2009-08-02 19:29:15 -04:00
David Sauve dab6a32c07 Small update to XHValueRangeProcessor to ensure it also works with flot values 2009-08-02 14:38:33 -04:00
David Sauve 793e878cd3 Better value range processing. Now using a custom value range processor. 2009-08-02 14:34:24 -04:00
David Sauve 53c5a674ea Better value range processing? Custom value range processor. Not working with integers though... 2009-08-02 14:18:40 -04:00
David Sauve 71f7b0ad08 Removed debug text 2009-08-02 10:41:58 -04:00
David Sauve 724d15bc96 Number value ranges working 2009-08-02 10:24:44 -04:00
David Sauve 55ed0f4359 Encoding int, long, and float values when storing in values 2009-08-01 20:51:54 -04:00
David Sauve 966e42424c Debugging value range processor 2009-08-01 20:20:32 -04:00
David Sauve 234dcfe312 Formatting change 2009-08-01 20:11:53 -04:00
David Sauve 74376185ba Better term generation 2009-08-01 16:18:07 -04:00
David Sauve d60dc5a12b A better _from_pythong method 2009-08-01 16:15:21 -04:00
David Sauve 685b4edc19 Revert "Added value range processors. Not working yet."
This reverts commit d9a10eac95.
2009-08-01 14:08:44 -04:00
David Sauve d9a10eac95 Added value range processors. Not working yet. 2009-08-01 14:08:30 -04:00
David Sauve 98a6d5ea95 Added value range processors. Not working yet. 2009-08-01 14:07:36 -04:00
David Sauve c834a319e7 Finished refactor. Code should be much easier to work with now. 2009-07-31 15:29:48 -04:00
David Sauve edd41f4552 Fixed highlighting in newly refactored code 2009-07-31 15:23:55 -04:00
David Sauve 2a9aea6fa6 Revert "Fixed facetting in refactor branch"
This reverts commit 687b48f614.
2009-07-31 15:18:18 -04:00
David Sauve 687b48f614 Fixed facetting in refactor branch 2009-07-31 15:18:11 -04:00
David Sauve 9c986458b3 Fixed facetting in refactor branch 2009-07-31 15:17:22 -04:00
David Sauve 8ee12dd68d Continued refactoring. 23 of 27 tests passing now. Code is much cleaner and leaner 2009-07-31 11:46:14 -04:00
David Sauve 7752a6e743 Implemented _dump_document_data and new update method 2009-07-31 11:43:58 -04:00
David Sauve 856777adb4 Completed new _build_schema and _prepare methods 2009-07-31 11:43:14 -04:00
David Sauve 91043533ed Corrected year in copyright notice. Should have been 2009, not 2007 2009-07-30 09:01:38 -04:00
David Sauve 3f97fa28b0 Added copyright notices 2009-07-30 08:57:35 -04:00
David Sauve d4c8c42275 Fixed broken NOT query -- Thanks Supreet 2009-07-30 08:54:46 -04:00
David Sauve c087348a48 Implemented highlighting 2009-07-27 15:12:20 -04:00
David Sauve 224cd0855b Changed clear to use private _get_enquire method 2009-07-24 16:41:40 -04:00
David Sauve 199982fb33 Revert "Moved enquire instance creation to seperate private _get_enquire method"
This reverts commit 11d1213c33.
2009-07-24 16:15:34 -04:00
David Sauve 11d1213c33 Moved enquire instance creation to seperate private _get_enquire method 2009-07-24 16:15:24 -04:00
David Sauve 8fb2bca7d0 Moved enquire instance creation to seperate private _get_enquire method 2009-07-24 16:05:31 -04:00
David Sauve 4de6161ffd Moved query parser setup to _get_query_parser private method 2009-07-24 15:27:43 -04:00
David Sauve c9c831a332 Refactored search code to have a seperate private method for _get_flags and _get_sorter. Makes the search method a little easier to follow. 2009-07-22 16:47:30 -04:00
David Sauve b4931e1d43 Changed date generation in _from_python method to not including formatting 2009-07-22 11:30:22 -04:00
David Sauve 0c27d0e8cf Changed boolean values in _from_python to be t or f instead of true or false. Save a little bit of space 2009-07-21 13:47:13 -04:00
David Sauve 979f0f86a3 Revert "Refactored indexer creation into seperate private method _get_indexer"
This reverts commit b1c1a35ece.
2009-07-21 13:45:17 -04:00
David Sauve b1c1a35ece Refactored indexer creation into seperate private method _get_indexer 2009-07-21 13:45:06 -04:00
David Sauve b3ba9233e5 Refactored indexer creation into seperate private method _get_indexer 2009-07-21 13:44:50 -04:00
David Sauve 317fb6cd3d Moved database opening into private method 2009-07-21 13:11:33 -04:00
David Sauve 5a71dc76e2 Whitespace change 2009-07-21 09:43:55 -04:00
David Sauve ff0a466241 Implemented multi-field ordering on search 2009-07-09 16:35:24 -04:00
David Sauve 00892d695e Fixed sorting by integer and added tests to prove it 2009-07-09 14:04:06 -04:00
David Sauve 1a35a91a47 Implemented order by for single field 2009-07-09 13:00:07 -04:00
David Sauve b147b5c926 Roughed in the order_by 2009-07-08 16:43:46 -04:00
David Sauve 12e494e747 Fixed a bug that caused indexed=False to be ignored again after a previous refactor 2009-07-03 16:03:44 -04:00
David Sauve f8f453ad58 Moved _build_schema to bottom of class and commented it. Also moved unified schema method to inside. 2009-07-03 15:50:50 -04:00
David Sauve d43f7d1093 Better schema generation. Dropping the stuff I don't need, and including value column locations 2009-07-03 15:44:01 -04:00
David Sauve 0af2285c6f Updated term generation to not include fields marked as indexed=False 2009-07-03 15:18:23 -04:00
David Sauve 9e6912541e Refactored to use existing site.build_unified_schema() instead of generating this manually 2009-07-03 14:34:43 -04:00
David Sauve f77b861f81 Another refactor to further simplify narrowing 2009-07-03 11:55:25 -04:00
David Sauve 26df069e76 Refactored narrowing to not weight based on narrow query. ie., to only filter. 2009-07-03 11:53:03 -04:00
David Sauve b937536575 Implemented 2009-07-03 11:21:07 -04:00
David Sauve 061703134a Removed clean method. Is now implemented in the BaseSearchBackend 2009-06-25 08:51:31 -04:00
David Sauve bb5cbbd606 Small refactor on spelling suggestions in SearchBackend.search method 2009-06-20 07:34:26 -04:00
David Sauve 0c0be221cb Added spelling suggestions to SearchBackend.search method 2009-06-20 07:32:18 -04:00
David Sauve 7cd5335df5 Hide the highlight not implemented warning when highlight is False 2009-06-18 13:22:39 -04:00
David Sauve 9cb2305a61 Fixed SearchBackend.update not updating existing documents, but instead appending to index 2009-06-18 12:31:47 -04:00
David Sauve 0d77e02d80 Added a ton of docstrings and some small refactoring 2009-06-18 12:15:13 -04:00
David Sauve 993b28155a Fixed more_like_this to omit the document used in the query 2009-06-17 19:30:25 -04:00
David Sauve fa02d65941 Implemented more_like_this 2009-06-17 16:54:39 -04:00
David Sauve 2c80f85ec6 Made stemmer an instance variable (default is english) and added corresponding tests 2009-06-17 15:20:52 -04:00
David Sauve 86a5050750 Initial commit 2009-06-16 14:48:11 -04:00