<-[[.:start]] ====== YaDDNet Data Backup ====== ** Offsite Text file backups of all DSC messages ** * Alan Spindel initially suggested an offsite backup of the DSC messages in 2019 * YaDDNet has had the facility to send copies of each new message out to specified IP addresses via UDP since that time * The DSC messages are sent by UDP as simple plain text, for easy archiving * The archived text file can be used if necessary to rebuild the SQL database table containing the DSC messages in case of disaster * Since I'm now hosting YaDDNet again I decided I should run a UDP log receiver at home to improve the backup provision * The daily archive files will be ''gzipped'' and stored on Dropbox * I could share the link to anyone who is interested in using the saved data ===== YaDDNet UDP Send to Mirror ===== * In the ''PyYadd.py'' function a DSC message is created from the raw incoming data from YaDD (and DSC Decoder) in a standard format. * This involves converting MMSI -> Coast or Ship name, finding the ''[MID]'' and corresponding country etc. * Prior to using the newly created message to inject new data into the SQL server ''PyYadd.py'' calls the function ''send_to_mirror()'' to send out the UDP packet to any configured IP addresses * ''PyYadd.py'' then inserts the new data in the SQL server * Any addressed & listening UDP clients receive the UDP packets and append them to a log file



def send_to_mirror(data):
    log_data = "[log];"+data
    port = 2505
    host = "aaa.bbb.ccc.ddd.eee"
    host2 = "fff.ggg.hhh.iii"
    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock.sendto(log_data, (host, port))
    sock.sendto(log_data, (host2, port))

    return

===== Backup UDP receive and logrotate ===== ==== UDP Listener ====



#!/usr/bin/env python
#
# udp_logger.py
#

import SocketServer
import socket
import threading

class ThreadedUDPRequestHandler(SocketServer.BaseRequestHandler):
   
    def handle(self):
        data = self.request[0].strip()
        socket = self.request[1]
        self.write_file(data)
       
    def write_file(self,text):
        filename = '/home/gm4slv/yaddlogs/yaddnet_udp_log.txt'
        f = open(filename, 'a+')
        log = text
        f.write(log+"\n")
        f.close()
        return
 
class ThreadedUDPServer(SocketServer.ThreadingMixIn, SocketServer.UDPServer):
    pass

if __name__ == "__main__":
    HOST, PORT = "", 2505
    server = ThreadedUDPServer((HOST, PORT), ThreadedUDPRequestHandler)
    ip, port = server.server_address
    server.serve_forever()
    # Start a thread with the server -- 
    # that thread will then start one
    # more thread for each request
    server_thread = threading.Thread(target=server.serve_forever)
    # Exit the server thread when the main thread terminates
    server_thread.daemon = True
    server_thread.start()
    server.shutdown()

* The UDP Log Receiver listens to UDP/2505 for incoming packets from YaDDNet. These are of the form:



[log];2024-04-19 08:43:06;NLD_MB_HF1;2187.5;SEL;002050480;COAST;205;Belgium;Ostende Radio;SAF;636092799;SHIP;636;Liberia;CISNES, D5HF4, Cargo;TEST;NOINF;--;--;REQ;OK
[log];2024-04-19 08:43:28;BDXC-Han-Remote;12577.0;SEL;002241022;COAST;224;Spain;Coruna Radio;SAF;538005991;SHIP;538;Marshall Islands;NAVIG8 VIOLETTE, V7IH3, Tanker;TEST;NOINF;--;--;REQ;OK

* each new packet is appended to the running log file ''/home/gm4slv/yaddlogs/yaddnet_udp_log.txt'' * the file will grow indefinitely if left to itself! ==== Logrotate === * run by Crontab every day at midnight * copies the current running log to a timestamped file * erases the contents of the running log file * ''gzips'' the timestamped copy and moves it to //Dropbox//



## CRONTAB ENTRY

00 00 * * * /home/gm4slv/bin/yaddlog_rotate.sh /dev/null 2>&1



#!/bin/bash
#
# yaddlog_rotate.sh

DROPBOX=/home/gm4slv/Dropbox/Yaddlogs/
DIR=/home/gm4slv/yaddlogs

INFILE=yaddnet_udp_log.txt

TIMESTAMP=`date +%Y%m%d_%H%M`

OUTFILE=$TIMESTAMP\_$INFILE

cp $DIR/$INFILE $DIR/$OUTFILE
cat /dev/null > $DIR/$INFILE


gzip $DIR/$OUTFILE

mv $DIR/$OUTFILE.gz $DROPBOX

* This creates a ''gzipped'' log file in my Dropbox of each day's new DSC messages === Starting the UDP Listener at boot-up === * crontab entry to start it as a background job at ''@reboot''


#
# CRONTAB ENTRY
#
@reboot python /home/gm4slv/bin/udp_logger.py &

--- //John Pumford-Green 19/04/24 09:31 BST// Last updated : ~~LASTMOD~~ ===== Further Information ===== [[https://www.yaddnet.org/index.php | YaDDNet]] {{tag>radio yaddnet}}