Carsten Müller

Carsten Müller

malison@hub.netzgemeinde.eu

This channel has not added a profile description yet

Ort:: Deutschland

Geschlecht:: Männlich

Fri, 15 Mar 2019 09:42:00 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

!Hubzilla Support Forum

I got some strange Errors on my primary account on hub.netzgemeinde.eu and the admin told me to post that in the forum

1. Various Friendica servers do not receive any comments from me, including those from @Stefan Münz and Produnis. A new delivery fails with the error message:

The requested URL /dreport/push/https://hub.netzgemeinde.eu/item/c634f1c6-be69-4feb-9400-7b91baae27c4 was not found on this server.

This might be a bug in Hubzilla.

2. With different Diaspora Pods my comments are duplicated, both get different ID's but only one of the comments is sent to Diaspora but the other one still appears in my stream.

3. Interesting is also that in a Hubzilla forum I see only a fraction of the comments under a post in my stream but when I look at the post as an original source I have the permission to see all the comments.

What I've done so far: Clear Cache and Cookies, relogin. Nothing happens. Can anyone confirm and help on this?

show all 38 comments

Fri, 15 Mar 2019 11:13:22 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

Right now commented a Diaspora Post: https://diasp.de/posts/f0536f1028cf01375c3c101b0e91c357 as my answer appears once, do you see the Hubzilla also double ?: https://hub.netzgemeinde.eu/display/f0536f1028cf01375c3c101b0e91c357

Interesting is that the comment was not twice but also at the delivery brought no results - it means that was not delivered. Only when he appeared twice later, the second and thus new comment was marked as Delivered.

That lets me assume that the second comment was taken by Diaspora and placed under a new Message ID on Netzgemeinde.eu, therefore the duplication.

Fri, 15 Mar 2019 12:29:08 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

Ok the same thing happens with friendica, single post here https://friendica.a-zwenkau.de/display//fdacc1ed-135c-8b64-05d8-bd7419400006 and duplicate posts in https://hub.netzgemeinde.eu/display/fdacc1ed-135c-8b64-05d8-bd7419400006

Fri, 15 Mar 2019 21:52:58 +0100 zuletzt bearbeitet: Fri, 15 Mar 2019 22:03:09 +0100

I take this opportunity to put "write the drawbacks of federating from ZOT to other protocols" on top of my toDo list. I am very sorry this has not been done yet, even though I feel it's very necessary.
Using federation outside ZOT is possible and works, but it comes with a few aspects to be considered, trafe off decisions to be taken. These need to be communicayed much clearer. Yes, that's a bug.
So this is just a quick and dirty attempt to sketch an answer.

The devs had to create a special privacy role "Social - federation" extra for federating from ZOT to other protocols. This role looses a few privacy settings that are more strict with any other role for no other reason but that ZOT has got many features to protect and respect your privacy that no other protocol knows, understands or has ever heard of.
It seems you ran into one of those drawbacks, as nomadicID can only and exclusively done with ZOT. The other systems simply don't know about it. To them you can only live at one address, and sometimes you may consider yourself lucky if they allow you to move house. But you want to live at several places at once. This is expected to confuse a few of the other systems. Hm. The address is xyz. Oh, no, it's abc. Oh. Hm. Eh. What to do now? Dunno. Well, whatever. I'll put the message out, it'll arrive somehow.
(...)

This is not Hubzilla's fault. Hubzilla's inventor has been laughed at as things like nomadic identity were claimed to be impossible to achieve. Well, with Hubzilla 4.0 we even got the first steps to nomadic content on top.

There is a reason that special privacy role had to be done against the dev's conviction to provide the best privacy aware tools there are, because the other protocols spew when they are facing many of ZOT's privacy aware "impossible" features. It's binary, either you decide to benefit from as much privacy as possible, then a ZOT only system is the way to go.
Federation to other protocols is a trade-off where you decide you want to let go many of Hubzilla's possibilities. I will take this opportunity and take care that this will be communicated much better in the future.

core:dev commits schrieb den folgenden Beitrag Wed, 20 Feb 2019 08:52:13 +0100

Merge branch 'dev' into 'dev'
Merge branch 'dev' into 'dev'

basic support for nomadic content, fix for permitted guests deleting their own files in cloud space of other channel,urlencode category widget links See merge request !1523

Fri, 15 Mar 2019 22:10:24 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

OK thanks this might be an answer to Problem 2. But what about the problems 1 and 3 in my post? It's really annoying when you comment on a post and don't know if it's transmitted and you can't resend because that wrong resend URL. And it's also annoying when you see just a few comments in your own stream and always have to take a look in the original post. My Hubzilla Admin says he doesn't know what he have to look for and I have to post my problem here.

Fri, 15 Mar 2019 22:25:19 +0100

1. You state you cloned. And want to federate to friendica which does not speak ZOT.
3. We are dealing with a distributed system where it is quite possible that comment b is taking one route and message a is taking another and maybe flirting with abgew hubs they meet along the way. Therefore it is quite possible that comment b even though posted at a later date may arrive earlier. Especially if there are a few hubs that are loaded, slow or whatever.
If this happens on the same hub though logs would be interesting. Without them I believe everything is just speculation.

Fri, 15 Mar 2019 22:31:48 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

1. Yes but to some federica server i can post normaly and to some i cant thats not a problem with ZOT thats a Server Problem. And what about the strange url in that resend link?
3. Yes im with you but why when the forum channel and i on the same server? And others on the same server see that comments in the forum channel

Fri, 15 Mar 2019 22:41:11 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

For questions about the logs we have to ask @Mark Nowiasz he is the Admin of that Hub.

Fri, 15 Mar 2019 22:48:53 +0100

Then I can only say, would someone who is more involved with friendica federation please step up?
:)

Fri, 15 Mar 2019 23:22:05 +0100

Mario Vavti

mario@hub.somaton.com

Add 1: the issue is fixed in https://framagit.org/hubzilla/core/merge_requests/1556 thanks for reporting.

Add 2: cloning and federating with services that do not understand nomadic locations (diaspora is one of them) does not work (not an Hubzilla issue).

Add 3: we would need logs of the forum and of the post owner to shed some light on this.

Sat, 16 Mar 2019 09:58:31 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

@Mario Vavti Thanx Mario, that helps a lot.

Add 1: No Problem :-) Do we have to address the delivery issues with each server admin individually?

Add 2: As I understood it: Because I switched my Primary Hub, comments on Diaspora and Friendica now take the old route over the old Primary - which still runs as a clone with me? And even though I've added the Friendica and Diaspora contacts in the new Primary too?

Add 3: The funny thing was: The comments were then published 24 hours later all at once and since the error has not surfaced again.

Wed, 20 Mar 2019 12:59:34 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

@Mario Vavti
Today I noticed that other comments on Diaspora accounts are doubled, not just my own. However, that only seems to be so with me because I have a user addressed and he does not see his own comments twice under the posts. It's kind of confusing. So i have to ask again for my understanding.

As I understood it: Because I switched my Primary Hub, comments on Diaspora and Friendica now take the old route over the old Primary - which still runs as a clone with me? And even though I've added the Friendica and Diaspora contacts in the new Primary too?

Wed, 20 Mar 2019 13:32:50 +0100

Mario Vavti

mario@hub.somaton.com

@Carsten Müller it is ompossible to know what is going exactly without logs of the actions.
Anyway, i repeat again federating with non-zot services and cloning is not compatible...

Wed, 20 Mar 2019 14:32:43 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

@Mario Vavti

it is ompossible to know what is going exactly without logs of the actions.

For questions about the logs we have to ask @Mark Nowiasz he is the Admin of that Hub. He said i have to ask in the forum and you said you want logs...

i repeat again federating with non-zot services and cloning is not compatible...

Yes i got that, thats not the question. Or are you going to tell me I should not continue to ask? I just want to understand how the interaction between Hubzilla and non-zot services works and why double posts appear at once. I can not be the only one who uses the cloning feature?

Wed, 20 Mar 2019 14:38:59 +0100

Mario Vavti

mario@hub.somaton.com

Or are you going to tell me I should not continue to ask?

No, ask please. I am just saying that it can have weird side-effects...

Anyway, there is an issue at framagit which might be related: https://framagit.org/hubzilla/core/issues/1349
Please report any findings there also...

Wed, 20 Mar 2019 14:47:40 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

Ok, I can narrow it down even further. The posts that are duplicated are all from users from hub.netzgemeinde.eu where my new Primary is. In another post by the same Diaspora user, someone from hubzilla.eu comments on his posts, I do not see them twice. AND. When I switch to my clone on hub.libranet.de I do not see the posts twice, neither my own nor those of other users.

Wed, 20 Mar 2019 14:56:52 +0100

Mark Nowiasz

buckaroo@hub.netzgemeinde.eu

Well, I can activate logging - but not forever, since the hub is rather a high-traffic site and furthermore I would need a hint what to look for.

Wed, 20 Mar 2019 15:08:04 +0100

Mario Vavti

mario@hub.somaton.com

@Mark Nowiasz i guess you would need to catch one of Carstens comments on a diaspora post and see what comes back relayed from the diaspora pod.

@Carsten Müller this only happens if you comment on diaspora posts right?
Possibly, since you are connected from both of your clones, diaspora treats that as two separate channels and relays the comment twice?

Wed, 20 Mar 2019 15:13:24 +0100

exactly the same thing happened to me a while ago. you can't be connected to diaspora from two different locations, otherwise all hell breaks loose. diaspora treats your channel from each loaction as a different contact, and your Hz channel does not know how or from where to talk to D*. in the end i had to throw away my Hz channel and create a new one, taking great care to connect only from one location to other protocols. means any clones i create are done without diaspora or other federating protocols.

Wed, 20 Mar 2019 15:17:20 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

@Mario Vavti

this only happens if you comment on diaspora posts right?

Yes different users, different pods but just diaspora

Possibly, since you are connected from both of your clones, diaspora treats that as two separate channels and relays the comment twice?

yes maybe. But the comment, no matter from whom, appears only once on the diaspora pod. and he appears only once in the clone, but twice in the primary. but as I said just with me and others of hub.netzgemeinde.eu but apparently only I see the double, Mark does not see his own comments double but I do. its realy weird

Wed, 20 Mar 2019 15:25:14 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

@phani (Hubzilla) no sorry thats not the answer, it has worked long before, and all things fine but from that moment i switched the primary from one hub to another things go crazy. and i dont think its a problem with diaspora or friendica because diaspora nor friendica doesnt have double posts. and the double ones have different message ids

I think it has to do with the fact that I changed my primary to clone and the clone to my primary

@Mario Vavti

this only happens if you comment on diaspora posts right?

no sorry it happens also to friendica

Wed, 20 Mar 2019 15:28:06 +0100

@Carsten Müller " no sorry thats not the answer..." -- well, i think it is. but take your own time to figure it out; have fun!

Wed, 20 Mar 2019 15:29:04 +0100

Mario Vavti

mario@hub.somaton.com

Yes, friendica communication goes over the diaspora protocol (except you connected via activitypub won't be better though i guess).

Wed, 20 Mar 2019 15:29:22 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

The strange thing is that it only happens to hub.netzgemeinde.eu. I dont see double posts on hub.libranet.de my former primary and dont get double posts from other users on other hubzilla servers

Wed, 20 Mar 2019 15:30:18 +0100

Mario Vavti

mario@hub.somaton.com

I think it has to do with the fact that I changed my primary to clone and the clone to my primary

Hm... But you also re-connected from your new primary right?

Wed, 20 Mar 2019 15:31:45 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

@phani (Hubzilla) and how do you explain then that I see also double posts from other hz user in diaspora posts but only those who are with me on the same server?

Wed, 20 Mar 2019 15:33:11 +0100

when it happened to me things were going wrong intermittently, not predictably, with increasing frequency. i believe it's Hz talking from different locations to different contacts that D* (and probably also friendica) consider different contacts but Hz believes to be one and the same.

but i certainly don't know everything; just had a very similar experience. since then zot / zot6 protocol has changed, but i keep seeing reminders not to use cloned channels for federation.

Wed, 20 Mar 2019 15:35:36 +0100

and how do you explain then that I see also double posts from other hz user in diaspora posts but only those who are with me on the same server?

because your different locations, primary or otherwise, are treated as different contacts by D* and FCA. receiving comments from those, on posts or comments that for Hz come from the same channel, produces unpredictable results.

Wed, 20 Mar 2019 15:40:04 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

@phani (Hubzilla) I understand what you say but if thats right, then the double posts have to be on diaspora or friendica not in hz. One post for each clone. That cant be the answer and its to simple to say: it produces unpredictable results
We have to look why it comes to thats results :-)

Wed, 20 Mar 2019 15:43:27 +0100 zuletzt bearbeitet: Wed, 20 Mar 2019 15:44:50 +0100

perhaps you have to. i went thru this whole thing earlier and am excusing myself from this thread now. "it produces unpredictable results" is exactly what i arrived at after several days & nights of headache when it happened to me. your result may be different of course....

Wed, 20 Mar 2019 15:55:22 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

@Mario Vavti

Hm... But you also re-connected from your new primary right?

Yes but weeks before i switched the primary status

Wed, 20 Mar 2019 15:59:18 +0100

Carsten Müller

malison@hub.netzgemeinde.eu

Maybe it has to do with 4.0? I dont know when @Mark Nowiasz updated the server?

Wed, 20 Mar 2019 21:09:07 +0100

This post doubling has been observed due to the fact that both Hubzilla and Friendica communicate with each other over the Diaspora protocol, but they can also communicate with each other over the ActivityPub protocol. Both protocols use different and incompatible message-id formats, which makes it difficult to detect duplicates (duplicates are detected using the message-id).

We have done some work in Hubzilla to detect these duplicates, but the last I heard Friendica still had not solved the problem on their end and this was causing untrackable duplicates on our end.

It was also later discovered in that the problem was magnified in subtle ways by the fact that both projects use both protocols. If only one project used both protocols, the problem could probably be solved easily. If both do, fixing it becomes orders of magnitude more difficult to solve. This is what I remember about the issue.

You may be able to fix it if *both* projects send the non-standard extension 'guid' field in ActivityPub and use this for duplicate checking across both protocol stacks (I believe this was created by Pleroma to resolve a similar post duplication issue between ActivityPub and OStatus). Hubzilla currently does not send or parse this. I don't know if Friendica does. If one were to add it, the declaration also needs to be added to the context schema so that signatures will still validate. This means it needs to be in the same context namespace on every project that uses it.

That's what I can tell you. Nobody is working on this at the moment and I no longer work on non-nomadic protocols, so the issue is sort of in limbo.