Support exclusion filters in beatmap search query #12148

notbakaneko · 2025-04-25T14:21:30Z

instead of #12134

fixes #12084
fixes #10916

Support exclusion filters. (-tag="blah blah")
Tags only exclude beatmapsets if all the beatmaps in the beatmapset contain the matching tag(s)

Keyword exclusions are full word only; bric will not exclude brick. Keyword exclusions also only excludes the more visible parts of the search results: title, artist, creator; it doesn't exclude fields that are likely to result in a high chance of unintended filtering.

Removes blindly feeding the keywords portion of the search query into simple_query_string and use proper match query that matches our query syntax, instead of trying to contort it into simple_query_string.

Moves the filters into a separate class.
Fix missing type hints and strict_types

rebased off Add Beatmapset search tests #12345
migrate relevant exclusion stuff from Better beatmap search query parsing #12134
Chose not to include the keyword parsing parts, as a query like cat -free might have unintended consequences of excluding all maps that have free any where in the searchable fields. Exclusions should explicitly use the -tag=free -title=free syntax to filter them out from the fields where they are not wanted.

Only tags support multiple include/exclusion at the moment; will look at updating the others later

nanaya · 2025-05-21T07:10:54Z

Also mentioned in #12134 but I think #12084 also includes excluding map tags which this PR doesn't handle?

notbakaneko · 2025-05-21T10:07:50Z

I recall it being mentioned mapper tags are usually used for titles, genres, etc; basically duplicating the existing fields but with extra variations on the names, so maybe exclude options should duplicate into the mapper tags as well 🤔

nanaya · 2025-05-21T10:14:34Z

general exclusion query currently doesn't work quite as expected anyway... which is supposed to be fixed by the closed pr (or my workaround pr or... just don't search user tags for general query?).

nanaya

still checking

nanaya · 2025-05-29T11:20:36Z

app/Libraries/Search/BeatmapsetSearchRequestParams.php

+        foreach ($parser->includes as $optionKey => $optionValue) {
            $propName = $optionMap[$optionKey] ?? null;

            if ($propName !== null) {
-                $this->$propName = $optionValue;
+                $this->includes->$propName = $optionValue;
+            }
+        }
+
+        foreach ($parser->excludes as $optionKey => $optionValue) {
+            $propName = $optionMap[$optionKey] ?? null;
+
+            if ($propName !== null) {
+                $this->excludes->$propName = $optionValue;


move this to BeatmapsetSearchOptions? And maybe set it right away when parsing the query?

nanaya · 2025-05-29T11:21:22Z

app/Libraries/Search/BeatmapsetQueryParser.php

        // Some locales have `,` as decimal separator.
        // Note that thousand separator is not (yet?) supported.
-        $value = str_replace(',', '.', $value);
+        $value = str_replace(',', '.', (string) $value);


can use update to strtr

nanaya · 2025-05-29T11:34:11Z

app/Libraries/Search/BeatmapsetSearch.php

-            ->filter(['term' => ['beatmaps.playmode' => Beatmap::MODES['mania']]]);
+        if ($this->excludes->keys !== null) {
+            // TODO: needs checking; exclude keys but only on mania maps
+            // TODO: this seems to make no sense to apply across a whole set?


no it doesn't. I think this should always imply filtering to just mania maps.

nanaya · 2025-05-29T11:40:26Z

app/Libraries/Search/BeatmapsetSearch.php

+        if ($include) {
+            $this->query->must($subQuery);
+        } else {
+            $this->nestedMustNot->should($subQuery);


doesn't seem to give correct result when combined with simple keyword search

nanaya · 2025-05-29T11:42:25Z

app/Libraries/Search/BeatmapsetSearch.php

+            // require exact match for full tags, partial otherwise.
+            // "grid snap" - match grid AND snap in any order. e.g. snap/grid is allowed.
+            // ""grid snap"" - match anything with "grid snap".
+            // "geometric/grid snap" - explicitly match the tag.


I'd say just

partial search on zero or one quote (tag=thing, tag="thing")

exact search on two quotes (tag=""thing"")

nanaya · 2025-05-29T11:45:41Z

app/Libraries/Search/BeatmapsetSearchOptions.php

+
+declare(strict_types=1);
+
+namespace App\Libraries\Search;


maybe give the beatmap params related classes their own namespace

nanaya · 2025-05-29T12:19:06Z

app/Libraries/Search/BeatmapsetSearch.php

+            $this->query->filter(['terms' => ['track_id' => $trackIds]]);
+        }
+
+        if ($this->excludes->featuredArtist !== null) {


I'm not sure where it went wrong but this doesn't seem to work quite right (168 is a-one):

include filter works as expected

notbakaneko · 2025-08-01T03:51:38Z

Updating this with tests, etc on a different branch so this PR doesn't fill with random commits

notbakaneko · 2025-10-09T06:36:00Z

(more) rebasing and stuff

must_not matching fields, not what we had before

…set to match before excluding

notbakaneko added the area:search label Apr 25, 2025

notbakaneko self-assigned this Apr 25, 2025

notbakaneko force-pushed the feature/beatmapset-query-excludes branch 3 times, most recently from e5672e8 to 02750db Compare May 9, 2025 09:09

notbakaneko force-pushed the feature/beatmapset-query-excludes branch from 02750db to b1a66b9 Compare May 16, 2025 06:59

notbakaneko marked this pull request as ready for review May 16, 2025 07:46

notbakaneko mentioned this pull request May 19, 2025

Better beatmap search query parsing #12134

Closed

2 tasks

nanaya requested changes May 29, 2025

View reviewed changes

notbakaneko force-pushed the feature/beatmapset-query-excludes branch from d3b293e to 5dabc24 Compare July 25, 2025 04:31

peppy added this to @peppy's untitled project Jul 29, 2025

github-project-automation bot moved this to Next up in @peppy's untitled project Jul 29, 2025

peppy moved this from Next up to In Progress in @peppy's untitled project Jul 29, 2025

notbakaneko marked this pull request as draft August 1, 2025 03:50

notbakaneko force-pushed the feature/beatmapset-query-excludes branch 2 times, most recently from 0d74a21 to 3bc77f4 Compare August 12, 2025 06:31

notbakaneko mentioned this pull request Aug 12, 2025

Add Beatmapset search tests #12345

Merged

notbakaneko force-pushed the feature/beatmapset-query-excludes branch 2 times, most recently from 1438bb6 to 2c0a6f4 Compare August 26, 2025 02:53

notbakaneko mentioned this pull request Aug 26, 2025

Updated Beatmap search query parsing #12364

Draft

2 tasks

notbakaneko force-pushed the feature/beatmapset-query-excludes branch from 7003037 to 96ce084 Compare September 8, 2025 22:20

notbakaneko marked this pull request as ready for review September 9, 2025 12:27

notbakaneko mentioned this pull request Sep 22, 2025

Add the ability to hide anime/questionable backgrounds #12415

Open

notbakaneko marked this pull request as draft October 9, 2025 06:35

notbakaneko force-pushed the feature/beatmapset-query-excludes branch from 96ce084 to b09dd46 Compare November 5, 2025 06:34

notbakaneko added 29 commits December 1, 2025 16:24

seperating options

0784843

use filters from options object

88ee76a

apply include and exclude tags

e715c4e

apply exclude creator and text filters

38ad3ef

apply exclude difficulty

74c1904

fix exclusions on nested fields;

7a00893

must_not matching fields, not what we had before

move creator exclude to nested mustNot

d75ae2a

more exclusions

98871e7

move main query to class level

3250060

handle addSimpleFilters exclusions

2937c2c

preg_match explodes on null with strict_types

efcbd69

support excluding by ranked date

bece429

move nested must_not and comment

44e0f83

unneeded

0d2395d

type BoolQuery

59ff31e

use strtr; add test for equivalence parsing

10411a9

set mapping when setting options

dc4958a

set options directly when parsing

5b077ef

split dataprovider into more params

456bab4

remap the keys directly in switch-case

f893853

filtering for mania keys implies filtering for mania maps

53e2a76

wrong type

ed28664

don't incorrectly match against converts when excluding

76cf14e

apply exclusion to correct level

aba581e

Add various exclusion-supporting filter tests

601c831

Add tags test; update tag matching behaviour

ba08042

update tag exclusion behaviour to require all beatmaps of the beatmap…

5a7e746

…set to match before excluding

include more query string keyword tests

e6f37f7

replacing some usages of simple_query_string with match

797ef53

notbakaneko force-pushed the feature/beatmapset-query-excludes branch from edba3cb to 797ef53 Compare December 1, 2025 07:25

Support exclusion filters in beatmap search query #12148

Are you sure you want to change the base?

Support exclusion filters in beatmap search query #12148

Uh oh!

Conversation

notbakaneko commented Apr 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nanaya commented May 21, 2025

Uh oh!

notbakaneko commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nanaya commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nanaya left a comment

Choose a reason for hiding this comment

Uh oh!

nanaya May 29, 2025

Choose a reason for hiding this comment

Uh oh!

nanaya May 29, 2025

Choose a reason for hiding this comment

Uh oh!

nanaya May 29, 2025

Choose a reason for hiding this comment

Uh oh!

nanaya May 29, 2025

Choose a reason for hiding this comment

Uh oh!

nanaya May 29, 2025

Choose a reason for hiding this comment

Uh oh!

nanaya May 29, 2025

Choose a reason for hiding this comment

Uh oh!

nanaya May 29, 2025

Choose a reason for hiding this comment

Uh oh!

notbakaneko commented Aug 1, 2025

Uh oh!

notbakaneko commented Oct 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

notbakaneko commented Apr 25, 2025 •

edited

Loading

notbakaneko commented May 21, 2025 •

edited

Loading

nanaya commented May 21, 2025 •

edited

Loading