Testing my own private repo...

fmvalsera

Hi all,
trying to test my own private repo (working with v3.5.4) I found that it doesn't work and I'm not sure if I'm doing something wrong.

I have a master instance with the Repo App installed. Already I have create a specific login with public access enabled. And I have also uploaded a Test App defining the author and an release. Additionally I have checked the access from the browser with that login and I see the Test App listed in "Public access" section. Maintenance and Builder modes are off.

So far so good.

On the other hand I have started an slave instance, changing in "System" section the Application repository with my params (url, user/pwd, allows, keys... an so on). Maintenance and Builder modes are on.

After that, I visit the "Repository" section in the Admin area but nothing is shown. I have clicked on the reload button and "no results available" message persists. Checking the logs in the master instance shows me that the slave one is asking for the "data/auth" and "data/access" urls but no error is printed anywhere.

I have checked the code trying to find out further and I have found that the "r3/repo/repo_update_modules.go" file (line 24) is trying to load the remote list of modules but I'm not sure if it's doing it properly, since it's asking for 5 attributes from "module" relation but, checking the database schema, these are from "repo_module" relation.

Well, I don't know if this is a clue about what is happening or I misunderstanding the code but any help to configure a private repo is appreciated.

Regards.

gabriel

Hello,

Did you also add meta data for at least 1 language?

Meta data is required, because this is what is being shown in the repository. It also must be marked as Shown in store to be displayed.

The rest sounds about right. If you are still having issues, please post the error logs from the slave instance so I can have a look. On the master instance, you can also check if there is something in the logs - just in case you have an authentication issue.

Best regards,

Gabriel

fmvalsera

Thanks for you quick reply, Gabriel... You are right, I was missing the meta data as I thought it was an optional info.

Now the slave instance is getting the remote application list. I'll continue the path towards my private repo and I let you know if I have any issue.

Best regards!

fmvalsera

Hi all!
my applications in the Repo hosted at the master instance are now listed in the slave instance. The problem is that the slave is complaining about "signature could not be verified by any trusted public key" when I try to install any.

Forgetting the remote Repo at this moment, I have isolated this test just to clarify what is happening with the keys:

In the master I'm exporting an application after generating a private/public key pair. This screenshot shows it:

Then I store the private key in memory and I click on the export button getting the .rei3 file.
With the public key I go to the System section in the slave instance to add the new public key and apply the changes. Like this:

Finally I go to the Applications sections to add from file the .rei3 I got from the master instance (generated with the private key), but I get this message:

After increasing the log verbose these are the messages:

I have try it with v3.6 and 3.5.4 with no luck. I guess I'm missing something very easy to solve but I cannot find out what it is. If someone could give me some hint ...

Thanks in advance.
Regards.

gabriel

fmvalsera
Hello again 🙂

You described everything very well - it´s very easy to follow. The weird thing is that from your description, you are doing everything correctly. To go into some detail, R3 needs two things to validate an application:

The application must be signed at export with a private key. It does not matter what the original R3 instance is or what keys it knows - only what private key was used during export.
The receiving R3 instance checks the signature of the application to be installed against all known public keys. So as long as the correct public key (e. g. corresponding to the signing key from step 1) is installed in the target instance, the signature validates.

The version of R3 is currently not relevant for this process, as we have never needed to change it. If the signature validates but the app cannot be installed because of an older version of R3, a different error will be shown.

I would first try to eliminate user error, as copy-paste errors can always occur. I would generate a new key pair, re-export the app with the new private key and use the new public key for the target instance.

If you are still having this issue, I can offer to test it on our end. If your test app does not have any secrets, you could send it to me together with your public key. If there is a bug in the signature validation, I can check it directly.

fmvalsera

Hi Gabriel, thanks for your invaluable help...

I will review again the process considering your comments. Meanwhile I attach the sample application (just a hello world app) and the public/private key pair.

Just for clarifying, I'm always using the same version (firstly 3.5.4 and lately 3.6) for both instances so they should be completely compatible for the test. On the other hand, both of them are running under a reverse proxy with nginx, but I think this is not relevant since there are no errors about URLs and everything seems to run right apart from the key validation which is executed locally in the target instance.

Thank you again. Best regards.

PS: I cannot attach the .rei3 file, the uploader is complaining about the extension. I have uploaded it to a github repo.

gabriel

Thank you for the file and keys. I´ve bootet up a test system and at first was not able to import. I checked it twice: The signature does not fit with your public key. My best guess would be that you used a different private key during the initial export or that there was a copy/past error.

But the good news is that the provided keys work. I did a small change to your app and re-exported it with your private key. I was then able to import your app successfully after I installed your public key.

Could you please try the same thing (update your app, re-export, install it) and let me know, if it works?

fmvalsera

Hi Gabriel,

I've tried what you suggested with no luck. But I have a new clue of what could be happening. Perhaps you know how to solve it.

In my environment REI3 is running under Docker with Linux at host, for both instances (source an target). I have tried the test several times but it doesn't work, even following your sequence.

After that I have started a portable REI3 instance on Windows. The application I created and exported from Linux I can import in Windows neither. The same error message is shown.

So I have recreated the application in the Windows instance with a new key pair. I have signed it in Windows with the private key and I have distributed the public one to the Linux target instance. Then I have imported successfully the application.

At this point, I guess that the problem is related to the signature that is generated with the private key in the source Linux instance. Could it be that the sign process (GO runtime, native libraries or other dependencies) does not produce the same signature depending on the operating system?

Well, I hope this helps to solve the issue because is the only difference I find.

Thanks for your time!

gabriel

fmvalsera
As long as you used the official releases (e.g. did not change the source code), there should not be any difference. But I will run tests to confirm. I´ll get back to you.

Sorry that you are having issues - its usually a pretty straightforward process.

gabriel

I´ve done some tests with 2 keys (one generated on a Windows machine, one generated on a Linux machine):

Both keys were newly generated. I´ve exported apps both from Windows & Linux systems with both keys and imported back into both Windows & Linux. All tests were successful.

The only thing I cannot test right now, is R3 under Docker. But they are the same binaries as the regular Linux release, so I struggle to come up with possible differences.

All dependencies for key handling (creation, signing, validation) are built-in to the binary as far as I know. There could potentially be differences between R3 releases (v3.5.4 / v.3.6 for example) because we upgrade the Go packages (net/crypto I think in this case) - but this would be the first time we had this issue.

I assume you have also tested new keys in your R3 instances on Docker? Just in case.

fmvalsera

Wow!, thank you so much for your help.

I continue testing and you are right, probably the issue is not related to the operating system.

I say that because the application generated in Windows (is also a dummy application but not exactly the first one I hosted at GitHub) can be imported in Linux. After that I exported again from Linux with the same key pair I used in Windows both .rei files are exactly the same. I mean the .json files inside are equals, including the signature attribute. And I can import the application with no problem.

So now I'm suspecting about the first application I created. As you told me in a previous reply you did a small change in the application, what was it? Is it possible that I'm missing some attribute or element necessary for exporting properly?

Sorry, I'm lost with this and I would like to check the reliability to distribute applications with the platform.

Thanks again.

fmvalsera

Well, I'm here again... And this time I think that I have found the problem.

For my first dummy application I set the "Builds on" application attribute to lsw_icons (the second one hadn't any dependency). If I clean that attribute the application can be exported/imported successfully in any environment. But if I set the dependency with lsw_icons, the export finishes ok but the import doesn't work.

Which is the right way to do it? I mean to link to another project. Of course I have installed the project lsw_tools from official Repo in the instance where I'm creating the dummy application (otherwise the combo "Builds on" wouldn't show the choice to add lsw_icons).

Thanks for helping me to check this.

gabriel

fmvalsera
Its valid to select other apps as "Builds on" - these apps will then be exported together with your app, because they have become dependencies.

However, as you and my colleague @lsw-fabian probably suspects, there could be an issue with their signatures. Usually you would not make any changes to apps from other authors - and if this is the case, they keep their original signature and will not be re-signed with your private key. Because lsw_icons was released by us, it uses the key REI3 official (2020). Is it possible that you removed the original trusted key for REI3? If so, then your app might be importable, but an app like lsw_icons cannot be verified.

lsw-fabian

Have you made any changes to lsw_icons?

fmvalsera

Hello,
I made no modificación to lsw_icons. As far as I understand the application exportation has no embedded dependencies. Since the target instance has no preinstalled dependencies neither, the application import fail.
The confusing was the received message about no valid public key on target instance while importing, rather than warning about missing dependency.
Well, I'm not sure if it's working this way... but it's my conclusion after all.
Thank you for your support.
Regards.

gabriel

fmvalsera
Bad timing, I have not seen your post before posting mine 🙂

When you Build on other applications, they become dependencies and they will automatically be exported as well. See my previous post for how this can cause an issue, regarding the apps having different signature keys.

fmvalsera

All right, now everything is clear. That's the point, I thought that the whole application was signed with the same private key.
Now I understand that the lws_icons keeps its original signature, so that's why I must add all the involved public keys in the target instance.
Thank you very much, both of you, for your five stars support during the weekend!!
Best regards.

gabriel

fmvalsera
And thank you for the excellent explanation and willingness to test. I´ll make a note to extend our documentation regarding application transfer - I don´t think we have all that detail in there yet.