Page 3 of 4

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Fri Jun 21, 2024 8:49 pm
by G7OMN
Oddly my S0 data didn't have text colour definitions for slot1, only slot 2 (go figure?).
You can easily fix this by duplicating the logic for TS1, with the appropriate changes in variables.
The next step is to try and increase the cap on the number of users read in from the nextionUser.csv or perhaps get it to automatically download, process and read in the data from kf5iw.com
A revised NextionDriver has been posted that increases the limit, but it has not made its way into the appropriate Pi-Star libraries:
<https://github.com/on7lds/NextionDriver ... bf273dcb08>

[/quote]

I didn't need to worry about slot 1 in the end - when i checked again, i had mistakenly removed it, ahead of making the comment.
I used your code anyway.

I spotted the changes on ON7LDS' Git last night - it's an amendment to the NextionDriver.h file.
I cloned the repo to the pi in the hope with the intention of recompiling it - but it has failed with errors. So I need a bit more time to look at it.

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Fri Jun 21, 2024 9:41 pm
by KN2TOD
Try this:

Code: Select all

rpi-rw
cd /tmp
wget "https://github.com/on7lds/NextionDriver/archive/refs/heads/master.zip"
sudo unzip master.zip
cd NextionDriver-master
sudo make -f Makefile
/tmp/NextionDriver-master/NextionDriver -V
You'll get a couple of warning messages from the compile, but they can (safely) be ignored. (The last command in the sequence above does a basic "test" of the resulting binary, to make sure it can at least be executed.)

(Note: I also modified the NextionDriver.h file to bump the version from 1.25 to 1.26. This was NOT changed in the downloaded code.)

Rename the old driver and move in the new driver, and then restart the driver (or simply reboot):

Code: Select all

rpi-rw
sudo mv /usr/local/bin/NextionDriver /usr/local/bin/NextionDriver-old
sudo cp /tmp/NextionDriver-master/NextionDriver /usr/local/bin
/usr/local/bin/NextionDriver -V
sudo systemctl restart nextiondriver.service
grep -ashie "NextionDriver: " {/var/log/syslog.1,/var/log/syslog}
The last command above should show you a "successful" restart of the driver (messages extracted from the system log), with the expected increase in the limits:

Code: Select all

Jun 21 16:30:29 pi-star-nvme NextionDriver: NextionDriver version 1.26
Jun 21 16:30:29 pi-star-nvme NextionDriver: Copyright (C) 2017...2021 ON7LDS. All rights reserved.
Jun 21 16:30:29 pi-star-nvme NextionDriver: Starting with verbose level 2
Jun 21 16:30:29 pi-star-nvme NextionDriver: Reading configuration file /etc/mmdvmhost
Jun 21 16:30:29 pi-star-nvme NextionDriver: Found RX Frequency 430450000
Jun 21 16:30:29 pi-star-nvme NextionDriver: Found TX Frequency 439450000
Jun 21 16:30:29 pi-star-nvme NextionDriver: Found Location [Atlanta GA]
Jun 21 16:30:29 pi-star-nvme NextionDriver: Use Transparent Connection: YES
Jun 21 16:30:29 pi-star-nvme NextionDriver:   Send Frame Type: YES
Jun 21 16:30:29 pi-star-nvme NextionDriver:   Local port: 40094
Jun 21 16:30:29 pi-star-nvme NextionDriver:   Remote port: 40095
Jun 21 16:30:29 pi-star-nvme NextionDriver: Found Virtual Port [/dev/ttyAMA10]
Jun 21 16:30:29 pi-star-nvme NextionDriver: Found Nextion Port [modem]
Jun 21 16:30:29 pi-star-nvme NextionDriver: RemoveDim: ON
Jun 21 16:30:29 pi-star-nvme NextionDriver: WaitForLan: ON
Jun 21 16:30:29 pi-star-nvme NextionDriver: SleepWhenInactive: OFF
Jun 21 16:30:29 pi-star-nvme NextionDriver: Using verbose level 2
Jun 21 16:30:29 pi-star-nvme NextionDriver: Running on Raspbian GNU/Linux 12 (bookworm)
Jun 21 16:30:29 pi-star-nvme NextionDriver: Pi-Star v 4.3.0
Jun 21 16:30:29 pi-star-nvme NextionDriver: Opening ports
Jun 21 16:30:29 pi-star-nvme NextionDriver:  /dev/ttyAMA10 (=/dev/pts/0) <=> modem
Jun 21 16:30:29 pi-star-nvme NextionDriver: Data files directory: /usr/local/etc/
Jun 21 16:30:29 pi-star-nvme NextionDriver:  Groups file : 2024-06-21 04:25:07 (12 hour old)
Jun 21 16:30:29 pi-star-nvme NextionDriver:  Users file : 2024-06-21 04:25:07 (12 hour old)
Jun 21 16:30:29 pi-star-nvme NextionDriver:   Reading groups from /usr/local/etc/groups.txt
Jun 21 16:30:29 pi-star-nvme NextionDriver:   Read 1634 groups.
Jun 21 16:30:29 pi-star-nvme NextionDriver:   Reading users from /usr/local/etc/stripped.csv
Jun 21 16:30:29 pi-star-nvme NextionDriver:    delimiter ','
Jun 21 16:30:29 pi-star-nvme NextionDriver:    DMRid in field 1
Jun 21 16:30:29 pi-star-nvme NextionDriver:    Call  in field 2
Jun 21 16:30:29 pi-star-nvme NextionDriver:    Name  in fields 3 + 4
Jun 21 16:30:29 pi-star-nvme NextionDriver:    Extra data in fields 5,6 and 7
Jun 21 16:30:30 pi-star-nvme NextionDriver:   Read 261531 users in 400 ms.
Jun 21 16:30:30 pi-star-nvme NextionDriver:   Sorted CALL table in 289 ms.
Jun 21 16:30:30 pi-star-nvme NextionDriver: Disk size : 7124 MB (3976 free)
Jun 21 16:30:30 pi-star-nvme NextionDriver: Started with screenLayout 0
Jun 21 16:30:30 pi-star-nvme NextionDriver: Started with verbose level 2
Jun 21 16:30:30 pi-star-nvme NextionDriver: Dim commands will be removed
Jun 21 16:30:30 pi-star-nvme NextionDriver: Opening sockets ...
Jun 21 16:30:30 pi-star-nvme NextionDriver: Transparent Connection: talking socket open, fd=2
Jun 21 16:30:30 pi-star-nvme NextionDriver: Try to bind 0.0.0.0 ...
Jun 21 16:30:30 pi-star-nvme NextionDriver: Transparent Connection: listening socket open, fd=3
Jun 21 16:30:30 pi-star-nvme NextionDriver: Transparent data sockets active
Jun 21 16:30:30 pi-star-nvme NextionDriver: I can not (yet) check or update modem connected displays
Jun 21 16:30:31 pi-star-nvme NextionDriver: Starting with network interface eth0:192.168.1.124

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Sat Jun 22, 2024 9:17 am
by G7OMN
Thanks...
That was the same set of errors I got when i compiled, so i went around again.
I updated the version to 1.26 as you did, but set the max users limit to 350K, so i won't need to go around again for a good while.

I see you are still using a stripped.csv for the users.
Are you manually updating or do you have it pulling from somewhere automatically?

It looks like it should be able to get the csv and process from a web location. So i tried this yesterday by dropping a file on a host and altering the DMRidFileSrc address, but it didn't see to change when i rebooted or forced an update in Pistar. I guess during the nightly house keeping should be the time it does this, but i would have thought a forced update does the same?
I looked through the chron.d and pi-star daily routines, and there's nothing to suggest this is done - however i can see that the DMRIds.dat file is updated daily and i couldn't see a ref to this either.

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Sat Jun 22, 2024 3:59 pm
by KN2TOD
G7OMN wrote: Sat Jun 22, 2024 9:17 am I see you are still using a stripped.csv for the users.
Are you manually updating or do you have it pulling from somewhere automatically?
A download for nextionUsers.csv has been added to the latest releases; you should be able to substitute that file for stripped.csv in the configuration for the Nextion driver. (I just haven't got round to changing my config to match.)

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Sun Jun 23, 2024 1:57 pm
by G7OMN
KN2TOD wrote: Sat Jun 22, 2024 3:59 pm
G7OMN wrote: Sat Jun 22, 2024 9:17 am I see you are still using a stripped.csv for the users.
Are you manually updating or do you have it pulling from somewhere automatically?
A download for nextionUsers.csv has been added to the latest releases; you should be able to substitute that file for stripped.csv in the configuration for the Nextion driver. (I just haven't got round to changing my config to match.)
I'm not sure how this is supposed to work.
It doesn't update on manual update/upgrade.
It doesn't update on restart
and it doesn't update in nightly updates either - despite pointing to another URI, it's still the same filesize and states 6 days old in the user.log.
The one I'm expecting it to import is about 3MB bigger.

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Sun Jun 23, 2024 3:57 pm
by KN2TOD
G7OMN wrote: Sun Jun 23, 2024 1:57 pm
KN2TOD wrote: Sat Jun 22, 2024 3:59 pm A download for nextionUsers.csv has been added to the latest releases; you should be able to substitute that file for stripped.csv in the configuration for the Nextion driver. (I just haven't got round to changing my config to match.)
I'm not sure how this is supposed to work.
It doesn't update on manual update/upgrade.
It doesn't update on restart
and it doesn't update in nightly updates either - despite pointing to another URI, it's still the same filesize and states 6 days old in the user.log.
The one I'm expecting it to import is about 3MB bigger.
When you execute the pistar-update process *or* when you leave your HS on overnight, the HostFilesUpdate.sh gets executed. Buried in all the files it downloads is one for the nextionusers.csv.

It's actually a two stage process: it downloads a zip (.gz) file (about 4.5m) then unzips it (about 15m). The file can be found in the /usr/local/etc/ directory.

But here's the kicker: it only downloads the file IF the current file is 7 days or older.

If you want to manually update the file regardless of its age, you can execute this:

Code: Select all

rpi-rw
sudo curl -sSL http://www.pistar.uk/downloads/nextionUsers.csv.gz | gunzip -c > /tmp/nextionUsers.csv
sudo cp /tmp/nextionUsers.csv /usr/local/etc/nextionUsers.csv 
of more directly, this:

Code: Select all

rpi-rw
sudo ssh -c "curl -sSL http://www.pistar.uk/downloads/nextionUsers.csv.gz | gunzip -c > /usr/local/etc/nextionUsers.csv"

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Mon Jun 24, 2024 5:21 pm
by G7OMN
WRT to below, this poses three questions from me:
# Download Nextion Users
if [ -f ${NEXTIONUSERS} ]; then
if [[ $(find "${NEXTIONUSERS}" -mtime +7) ]]; then
curl -sSL http://www.pistar.uk/downloads/nextionUsers.csv.gz --user-agent "Pi-Star_${pistarCurVersion}" | gunzip -c > ${NEXTIONUSERS}
fi
else
curl -sSL http://www.pistar.uk/downloads/nextionUsers.csv.gz --user-agent "Pi-Star_${pistarCurVersion}" | gunzip -c > ${NEXTIONUSERS}
fi
1. It should be possible to reduce this to another interval by changing the -mtime +7 to something else, eg +2 to bring it to 2 days, if pulling it from another server (so as not to overload pistar.uk.)
2. I have changed my DMRidFileSrc address - it's not this value taken from the HostfilesUpdate.sh.... So is ${NEXTIONUSERS} the variable for the address and filename i have specified, and if that fails, it defaults to http://www.pistar.uk/downloads/nextionUsers.csv.gz?
3. or is http://www.pistar.uk/downloads/nextionUsers.csv.gz hardcoded as the source in this script??

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Mon Jun 24, 2024 9:27 pm
by KN2TOD
G7OMN wrote: 1. It should be possible to reduce this to another interval by changing the -mtime +7 to something else, eg +2 to bring it to 2 days, if pulling it from another server (so as not to overload pistar.uk.)
You can certainly make these changes if you want, but IF another update gets issued for Pi-Star, your changes will be backed out.
G7OMN wrote: 2. I have changed my DMRidFileSrc address - it's not this value taken from the HostfilesUpdate.sh.... So is ${NEXTIONUSERS} the variable for the address and filename i have specified, and if that fails, it defaults to http://www.pistar.uk/downloads/nextionUsers.csv.gz?
3. or is http://www.pistar.uk/downloads/nextionUsers.csv.gz hardcoded as the source in this script??
If I understand what you're asking here: no, you can not use the ${NEXTIONUSERS} variable in the Nextion Driver config. And, no, you can not use the "...csv.gz" file in the config. You have to reference the "/usr/local/etc/nextionUsers.csv" file.

The standard Nextion config looks like this:

Code: Select all

[NextionDriver]
Port=modem
LogLevel=2
DataFilesPath=/usr/local/etc/
GroupsFile=groups.txt
DMRidFile=stripped.csv 
DMRidDelimiter=,
DMRidId=1
DMRidCall=2
DMRidName=3,4
DMRidX1=5
DMRidX2=6
DMRidX3=7
RemoveDim=1
SleepWhenInactive=0
ShowModeStatus=0
WaitForLan=1
Change the DMRidFile= line to this (i.e. substitute "nextionUsers.csv" for "stripped.csv":

Code: Select all

  :
DMRidFile=nextionUsers.csv 
  :
I wouldn't worry too much about the "every 7 day" thing for the moment: only about 30-40 id's get added each day, on average, spread across the world. The chances of encountering one of these missing id's is slim. Even spotting 'em on the display, slimmer.

Having increased the driver's upper limit on id's, you've pretty much taken care of most of the MIA id's you'll probably encounter anyway, IMHO.

-----

I should mention than I'm using configs that are several years old that were restored into current images (overlaying what ever was there currently, by default). Looking at the Nextion config of one of the vanilla images, I notice that the config has added a few new params which is what you're seeing (and I've not using nor added to my configs):

Code: Select all

[NextionDriver]
Port=modem
LogLevel=2
DataFilesPath=/usr/local/etc/
GroupsFile=nextionGroups.txt
GroupsFileSrc=https://www.pistar.uk/downloads/groups.txt
DMRidFile=nextionUsers.csv
DMRidFileSrc=https://www.pistar.uk/downloads/nextionUsers.csv
DMRidDelimiter=,
DMRidId=1
DMRidCall=2
DMRidName=3
DMRidX1=5
DMRidX2=6
DMRidX3=7
RemoveDim=0
SleepWhenInactive=600
ShowModesStatus=0
WaitForLan=1
If this is what you're seeing, then you're good to go with this config. The only thing I would change is: "DMRidName=3,4" (adds last name to the mix).

I haven't looked at what the driver does with the alternate source directives, but I'm assuming that if it doesn't find a required file locally (..File=), then it goes out and downloads a new copy pointed to by the alternate source (..FileSrc=) statement.

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Mon Jun 24, 2024 9:55 pm
by G7OMN
the point i am making is the URI used in the mmdvmhost host file does not match what is in the HostfilesUpdate.sh.
My current mmdvmhost:
[NextionDriver]
Port=/dev/ttyUSB0
LogLevel=2
DataFilesPath=/usr/local/etc/
GroupsFile=nextionGroups.txt
GroupsFileSrc=https://www.pistar.uk/downloads/groups.txt
DMRidFile=nextionUsers.csv
DMRidFileSrc=https://duncreggan.net/pi-star/nextionUsers.csv
DMRidDelimiter=,
DMRidId=1
DMRidCall=2
DMRidName=3,4
DMRidX1=5
DMRidX2=6
DMRidX3=7
RemoveDim=0
SleepWhenInactive=0
ShowModesStatus=0
WaitForLan=1
Path from HostFilesUpdate.sh:
# Download Nextion Users
if [ -f ${NEXTIONUSERS} ]; then
if [[ $(find "${NEXTIONUSERS}" -mtime +7) ]]; then
curl -sSL http://www.pistar.uk/downloads/nextionUsers.csv.gz --user-agent "Pi-Star_${pistarCurVersion}" | gunzip -c > ${NEXTIONUSERS}
fi
else
curl -sSL http://www.pistar.uk/downloads/nextionUsers.csv.gz --user-agent "Pi-Star_${pistarCurVersion}" | gunzip -c > ${NEXTIONUSERS}
So which is it using to try and update?
That is why i asked if there was a variable it was using instead....

Re: v4.16 / 4.2.1 / v4.3.0 Nextion field mapping?

Posted: Mon Jun 24, 2024 11:36 pm
by KN2TOD
Further up in the file (HostFilesUpdate.sh), the variable is set:

Code: Select all

  :
NEXTIONGROUPS=/usr/local/etc/nextionGroups.txt
NEXTIONUSERS=/usr/local/etc/nextionUsers.csv
  :
# Download Nextion Groups
if [ -f ${NEXTIONGROUPS} ]; then
        # Update ${NEXTIONGROUPS}
        if [[ $(find "${NEXTIONGROUPS}" -mtime +7) ]]; then
                curl --fail -o ${NEXTIONGROUPS} -s http://www.pistar.uk/downloads/groups.txt --user-agent "Pi-Star_${pistarCurVersion}"
        fi
else
        # Get ${NEXTIONGROUPS}
        curl --fail -o ${NEXTIONGROUPS} -s http://www.pistar.uk/downloads/groups.txt --user-agent "Pi-Star_${pistarCurVersion}"
fi
# Download Nextion Users
if [ -f ${NEXTIONUSERS} ]; then
        if [[ $(find "${NEXTIONUSERS}" -mtime +7) ]]; then
                curl -sSL http://www.pistar.uk/downloads/nextionUsers.csv.gz --user-agent "Pi-Star_${pistarCurVersion}" | gunzip -c > ${NEXTIONUSERS}
        fi
else
        curl -sSL http://www.pistar.uk/downloads/nextionUsers.csv.gz --user-agent "Pi-Star_${pistarCurVersion}" | gunzip -c > ${NEXTIONUSERS}
fi
  :
So, as I mentioned before, here it's downloading a compressed version of the file and then unzipping it into its expected destination. But, as you've noticed, the Nextion driver config downloads the uncompressed version, but only IF can't find the above expected file loaded by HostFilesUpdate.sh

Why use the compressed source in HostFilesUpdate.sh? Why only download it every 7 days? We can only assume this is to "save a few bucks/millseconds" here and there. Then again, why is the Nextion driver coded to have alternate sources for groups and users? We can only assume to is meant to overcome/bypass setup/configuration errors, so that, at least, it has something to work with for the displays. Or, it's meant to allow for experimentation. Don't know.

The alternate URI's don't have to match anything in the HostFilesUpdate.sh script. You can use other sources. Maybe these sources fix up the list: abbreviate country names, fill in missing fields, weed out miscreants, whatever. As long as they match up field-wise, using .csv style conventions, anything goes here.

And not everyone has these extra statements in their Nextion config anyway (like me), so there! :)