https://projects.osmocom.org/https://projects.osmocom.org/favicon.ico?16647414092020-04-19T08:22:52ZOpen Source Mobile CommunicationspySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=180272020-04-19T08:22:52Zlaforge
<ul></ul><p><a class="user active" href="https://projects.osmocom.org/users/15">dexter</a> what about this, it has been 3 months? at least a one-line input/opinion from you would be much appreciated.</p> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=180472020-04-20T18:58:36Zdexter
<ul></ul><p>I had a look at it, first of all the current master of pysim and sysmo-usim-tool test both fine.</p>
<p>From time to time we see problems when sysmo-usim-tool does changes to the file system which then cause the tests of pysim to fail and vice versa. At least in my opinion this wasn't to much of a problem yet, but I agree, the way we test is a bit clumsy.</p>
<p>When I get you right you want to use the APIs provided by pysim to check on the contents of the files and implement those checks as python unit-tests? Maybe we could also make pySim-read.py a bit more scripting friendly. Maybe have commandline arguments to specify which file/parameter to read out. Then we would also have some kind of separation and at the same time we woule improve pySim-read.py so that it can be used from scripts.</p> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=181802020-05-06T15:52:25Zfixeria
<ul></ul><p>One more change related to this ticket: <a class="external" href="https://gerrit.osmocom.org/c/pysim/+/18073">https://gerrit.osmocom.org/c/pysim/+/18073</a>.</p> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=182172020-05-09T15:20:12Zlaforge
<ul><li><strong>Priority</strong> changed from <i>Normal</i> to <i>High</i></li></ul> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=182182020-05-09T15:23:52Zlaforge
<ul></ul><p>I'm sorry, but this is starting to seriously impede the development of pySim.</p>
<p>From my point of view, what is needed is to properly reset the system to a sane state after executing tests. "system" here includes the contents of the SIM cards.</p>
So if a patch makes some changes to the filesystem, fine - we will then have to re-write the entire filesystem with known data before starting another test. This should work
<ul>
<li>for user-writable files on all card types</li>
<li>for ADM1-writable files on all card types</li>
</ul>
<p>If some files cannot be written even with ADM1, this shouldn't be a problem - it is safe to assume that any [future] patch we test will then also not be able to write those files.</p> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=182512020-05-12T09:47:48Zdexter
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>40</i></li></ul><p>The resetting of the file system contents should take place before the test executes. My idea is to use the erase() method in cards.py. I think at the moment this method is only programmed for one card. What I plan to do is to reset the contents of the files that pysim writes to 0xff, so that they are basically uninitalized. We can do this with a separate tool pySim-erase.py, this way we do not have to touch pySim-prog.py. The erase() method would use functions that do a read-after-write verification of the file contents, so that we can be sure that the card really has been erased. I already have a draft implementation of pySim-erase.py and some methods for the cards class that can do an erase of binary files and records in one line, so filling the erase() method for each card should be easy. For sysmo-isim-sjs2 I already filled the erase() method.</p>
<p>The following patches are in review now:<br /><a class="external" href="https://gerrit.osmocom.org/c/pysim/+/18205">https://gerrit.osmocom.org/c/pysim/+/18205</a> cards: remove len calculation<br /><a class="external" href="https://gerrit.osmocom.org/c/pysim/+/18206">https://gerrit.osmocom.org/c/pysim/+/18206</a> cards: reset uninitalized EF.AD<br /><a class="external" href="https://gerrit.osmocom.org/c/pysim/+/18207">https://gerrit.osmocom.org/c/pysim/+/18207</a> pysim-prog: move ADM sanitation to utils.py<br /><a class="external" href="https://gerrit.osmocom.org/c/pysim/+/18208">https://gerrit.osmocom.org/c/pysim/+/18208</a> utils: do not crash when all bytes of EF.IMSI are 0xFF<br /><a class="external" href="https://gerrit.osmocom.org/c/pysim/+/18209">https://gerrit.osmocom.org/c/pysim/+/18209</a> commands: add method to determine size of a non record oriented file<br /><a class="external" href="https://gerrit.osmocom.org/c/pysim/+/18211">https://gerrit.osmocom.org/c/pysim/+/18211</a> commands: add features to verify data written to files</p> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=182722020-05-12T12:10:25Zlaforge
<ul></ul><p>dexter wrote:</p>
<blockquote>
<p>The resetting of the file system contents should take place before the test executes. My idea is to use the erase() method in cards.py. I think at the moment this method is only programmed for one card. What I plan to do is to reset the contents of the files that pysim writes to 0xff, so that they are basically uninitalized.</p>
</blockquote>
<p>The problem is that this will probably break once somebody submits code for a new file to gerrit, which is not yet understood by pysim master. This way the card contents changes juts because we tried to verify any random submitted but not yet approved patch.</p>
<p>Hence I think it would be better to really do a full dump / backup of all files on each of the cards once, and then restore that backup before each test run. The "Backup" part is basically what "osmo-sim-test -o" from libosmocore/utils is generating. Maybe all that's needed is the inverse part, i.e. restoring such a file/directory based backup?</p>
<blockquote>
<p>We can do this with a separate tool pySim-erase.py, this way we do not have to touch pySim-prog.py.</p>
</blockquote>
<p>I agree this test pre-setup should be kept out of pySim-prog. Whether it's C code or python, I don't really have a preference.</p>
<blockquote>
<p>The erase() method would use functions that do a read-after-write verification of the file contents, so that we can be sure that the card really has been erased.</p>
</blockquote>
<p>If you want to stick with erase rather than restore, you should make sure to generate a list of all files/paths on each of the cards once, and then erase all their contents [assuming one can do this with ADM1 pin on all relevant cards]</p> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=183022020-05-12T15:09:42Zfixeria
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-7 priority-2 priority-default" href="/issues/4384">Feature #4384</a>: Jenkins program-read verification on daily (or weekly) basis</i> added</li></ul> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=186002020-06-08T08:47:53Zdexter
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Stalled</i></li></ul> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=218282021-04-05T17:28:12Zfixeria
<ul></ul><p>The build verification is broken again. It looks like one of the recent pySim patches has changed something on the test card(s):</p>
<p><a class="external" href="https://gerrit.osmocom.org/c/pysim/+/23631">https://gerrit.osmocom.org/c/pysim/+/23631</a> Use zero padding for EF['ACC'] field<br /><a class="external" href="https://gerrit.osmocom.org/c/pysim/+/23632">https://gerrit.osmocom.org/c/pysim/+/23632</a> Use zero padding for EF['ACC'] field</p>
<p>I tried to upload another patch that does not touch the actual parsing logic, and it fails too:</p>
<p><a class="external" href="https://gerrit.osmocom.org/c/pysim/+/23633">https://gerrit.osmocom.org/c/pysim/+/23633</a> Py2 -> Py3: do not inherit classes from object</p> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=218372021-04-06T11:11:22Zdexter
<ul></ul><p>Hello <a class="user active" href="https://projects.osmocom.org/users/67">fixeria</a></p>
<p>There is indeed something changed on the card:<br />Administrative data: 00000002<br /> MS operation mode: normal operation</p>
<p>is now:</p>
<p>Administrative data: 80000002<br /> MS operation mode: type approval operations</p>
<p>This is a bit odd, at least the patches I added recently do not touch this field. I will take care of this after lunch.</p> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=218492021-04-06T16:58:40Zdexter
<ul></ul><p>I have now fixed the problem by resetting EF.AD to its original state.</p> pySim - Bug #4383: Jenkins build verification is non-deterministichttps://projects.osmocom.org/issues/4383?journal_id=244562022-07-23T10:53:00Zlaforge
<ul><li><strong>Related to</strong> <i><a class="issue tracker-2 status-1 priority-2 priority-default" href="/issues/5127">Feature #5127</a>: tool for "re-formatting" [sysmocom] cards in tester</i> added</li></ul>