Question regarding per-connection content filters

Wed, 22 Feb 2023 14:30:25 +0100

adminsforum@hubzilla.org

Jupiter Rowland schrieb den folgenden Beitrag Wed, 22 Feb 2023 14:30:17 +0100

Question regarding per-connection content filters

I have a question regarding the content filter for connections which can be optionally activated.

Do they only filter posts out of my stream?

Or do they also remove comments and private messages? If this was the case, it'd be highly inconvenient, and I'd suggest at least an extra switch for it.

#Hubzilla #Connections #Filter

show all 17 comments

Wed, 22 Feb 2023 15:17:00 +0100

Mario Vavti

mario@hub.somaton.com

It is a pretty mighty tool and it all depends on how you configure it. See here for reference: https://framagit.org/hubzilla/core/-/blob/dev/Zotlabs/Lib/MessageFilter.php#L127

Sun, 26 Feb 2023 21:32:24 +0100

Jupiter Rowland

jupiter_rowland@hub.netzgemeinde.eu

We need documentation on that beyond the code where only devs know where to find what.

So I have two text fields per connection, a whitelist above, a blacklist below, which I see when editing the connection.

If I've understood this correctly, if I enter a word into the whitelist of a connection, everything that comes from this connection is deleted for me, whether it's a single post in my stream, a forum post, a comment or a private message.

Let's suppose I only want to see posts with certain words or hashtags, but only single posts, not forum posts, not comments and not private messages. Let's also suppose I'm neither a dev nor fluent in RegEx.

What exactly do I have to enter in the whitelist field?

Mon, 27 Feb 2023 08:06:33 +0100 zuletzt bearbeitet: Mon, 27 Feb 2023 11:16:46 +0100

Mario Vavti

mario@hub.somaton.com

@Jupiter Rowland huh... that is a tough call...

For the forum filter, you should probably filter the the forum channel to not send you items by author xy.

For the rest this should work:

?body ~= word1
?body ~= #hashtag
?item_private == 0
?item_thread_top == 1

Mon, 27 Feb 2023 09:46:20 +0100

RockyIII

rocky3@im.allmendenetz.de

@Jupiter Rowland

@Hans 4x? has also some tips:

https://federate.hopto.org/wiki/hans/Hubzilla_Tips/Filtering

Tue, 28 Feb 2023 10:34:47 +0100

Jupiter Rowland

jupiter_rowland@hub.netzgemeinde.eu

So if I have such a list of block patterns, it's safe to assume they're always connected with AND.

But now let's say that I want to implement a whitelist filter. Only posts containing at least one out of 29 words or hashtags are allowed to pass. However, this filter shall only be used on first posts in a thread, not on comments/replies, not on forum posts, not on personal messages.

Or do filters work in such a way that whitelist entries are alway connected with AND, and blacklist entries are alway connected with OR, regardless of format?

Tue, 28 Feb 2023 10:44:17 +0100

Mario Vavti

mario@hub.somaton.com

@Jupiter Rowland

However, this filter shall only be used on first posts in a thread, not on comments/replies, not on forum posts, not on personal messages.

An if/else syntax is not supported

Tue, 28 Feb 2023 12:51:58 +0100

Jupiter Rowland

jupiter_rowland@hub.netzgemeinde.eu

I don't mean an if/else syntax, only a query that involves both AND and OR. Not literally within the same query syntax, but the functionality.

Maybe this is already the case, albeit undocumented. Maybe this filter list

?body ~= word1
?body ~= #hashtag
?item_private == 0
?item_thread_top == 1

translates to a query like this: (?body ~= word1 OR ?body ~= #hashtag) AND ?item_private == 0 AND ?item_thread_top == 1

My situation is: I have a list of 34 words. I only want to see posts which contain at least one of these words. This is necessary because my stream would be cluttered with stuff I don't care for because it goes beyond the scope of this channel.

But: I only want first posts to be filtered this way. I still want to be able to see all replies/comments, all private messages etc.

The current situation is that if I put those 34 words into the whitelist of a contact, replies from that contact to my own posts that don't contain at least one of these 34 words are filtered. The same happens to private messages from contacts with the filter on, unless they contain one of these 34 words.

My goal is:

First posts from contacts shall be removed from my stream unless they contain one of thost 34 words.
Replies from the same contacts with the same filter shall not be blocked.
Private messages from the same contacts with the same filter shall not be blocked.

As it looks right now, if my assumption from above regarding the filter syntax was false, my current options would be:

I use the per-connection whitelist filter, and I can't see replies or private messages from these people.
I don't use the per-connection whitelist filter, I can see replies and private messages from these people, but my stream will be cluttered. It was cluttered before I had filters on, and it will be even more so now.

Whether or not this is the case, I currently consider Hubzilla's filter system severely flawed if it can only be used without nasty side-effects by people who a) can read code and b) know the Hubzilla source code. Whether it works under lab test conditions doesn't matter if it fails under real-life end-user conditions.

Thu, 02 Mar 2023 00:30:25 +0100

Jupiter Rowland

jupiter_rowland@hub.netzgemeinde.eu

Okay, I've tried a long list of

?body ~= <keyword or #hashtag>

entries, one of which contained two words, plus

?item_private == 0
?item_thread_top == 1

as the whitelist. As far as I could see, it didn't filter anything out of my stream.

Sat, 04 Mar 2023 11:21:10 +0100

Jupiter Rowland

jupiter_rowland@hub.netzgemeinde.eu

My next experiment will have only one ?body ≃ line with a regular expression. This should also allow me to shorten and at the same time expand the list.

Sat, 04 Mar 2023 11:29:37 +0100

Mario Vavti

mario@hub.somaton.com

@Jupiter Rowland regex will not work with ?body. Just place it in its own line...

Sat, 04 Mar 2023 11:39:48 +0100

Jupiter Rowland

jupiter_rowland@hub.netzgemeinde.eu

Thanks for telling me before the actual start of my experiment.

Sat, 04 Mar 2023 22:26:17 +0100

Jupiter Rowland

jupiter_rowland@hub.netzgemeinde.eu

Okay, so my experimental filter list looked like this:

/\b(List of words in the shape of a regular expression)\b/
?item_private == 0
?item_thread_top == 1

Result: Nothing is filtered at all.

Again: My goal is to only let posts through which

either are first posts in a non-private thread and contain at least one out of a list of n words
or are replies (= not first posts), regardless of content
or are private messages, regardless of content

Is that doable? If so, how?

Sat, 04 Mar 2023 22:36:41 +0100

Mario Vavti

mario@hub.somaton.com

@Jupiter Rowland

Is that doable?

No, as already mentioned there is no if syntax supported currently

Sat, 04 Mar 2023 22:52:48 +0100

Jupiter Rowland

jupiter_rowland@hub.netzgemeinde.eu

My understanding was that

?body ~= word1
?item_private == 0
?item_thread_top == 1

based on your suggestion here does exactly that, but only with one specific word instead of one out of a list of words.

What I wanted to do was replace word1 with a list of words.

My first experiment

?body ~= word1
?body ~= word2
?body ~= word3
?body ~= word4
?body ~= word5
?item_private == 0
?item_thread_top == 1

obviously didn't work because, to my best understanding, it would have only let posts through which contain all five words.

My next idea was

?body ~= /\b(word1|word2|word3|word4|word5)\b/
?item_private == 0
?item_thread_top == 1

which you said is impossible because ?body doesn't support regular expressions.

So I followed your suggestion and tried

/\b(word1|word2|word3|word4|word5)\b/
?item_private == 0
?item_thread_top == 1

This ended up not filtering anything either.

Basically, I have the choice between

using a word list
```
word1
word2
word3
word4
word5
```
or a regex
```
/(word1|word2|word3|word4|word5)/
```
but then all kinds of posts are filtered
letting all replies and private messages through, but first posts in public threads only when they contain one specific word, e.g. word1
not filtering at all

Sat, 04 Mar 2023 23:03:08 +0100

Mario Vavti

mario@hub.somaton.com

I mean you can not use different filter sets for toplevel posts, comments or limited posts currently because that would require an if clause...

Sat, 04 Mar 2023 23:12:11 +0100

Jupiter Rowland

jupiter_rowland@hub.netzgemeinde.eu

That wasn't what I wanted to do.

What I wanted to do was use one filter set, i.e. one word list, but only on top-level posts.

Maybe I should go to Framagit, open an issue and suggest either GUI switches for what shall be filtered or hard-coding Hubzilla to only filter top-level posts in the first place because that's where filters per connection make the most sense.

Sun, 05 Mar 2023 14:57:24 +0100

Mario Vavti

mario@hub.somaton.com

@Jupiter Rowland

What I wanted to do was use one filter set, i.e. one word list, but only on top-level posts.

This requires something like

if (toplevel post) {
    wordlist
}

It is certainly doable but not currently implemented...