Re: developing a data logger

Hmm, where to start? The internet should provide many examples of doing this, and there are even fairly well-written packages that do so, some of them in python. There are *lots* of ETL solutions available now, including the king of batch ingestion - Airflow (written in Python).

But, it depends on your sources of data. check for network or device monitoring tools if that's the sort of thing you are after. There are no doubt many packages out there to support IOT devices as well, generally built to scale better.

For the database interface - if you are using PostgreSQL - then check out psycopg (latest version is v3, NOT psycopg2, even if v2 still works...). If you prefer a more database agnostic approach, there is SqlAlchemy or indeed Django's ORM.

Might also be worth checking out time series data (aging, aggregating and coalescing data over time), something that PostgreSQL can support as well.

HTH, David

On Wed, Jul 12, 2023 at 8:23 PM o1bigtenor <o1bigtenor@gmail.com> wrote:

On Wed, Jul 12, 2023 at 3:38 AM David Nugent <davidn@uniquode.io> wrote:
>
> It depends on what you want to do. Simply gathering data and scooping it into a db does not require Django, which will probably not help much in this regard other than defining models/tables for your db and supporting migrations.
>
> If you want to provide web views or an API for this data, you're definitely in the right ball park.
>
>
Thank you very much for your assistance!!!

You are calling it 'gathering data and scooping it into a db' - - - -
this is the
part that I'm finding it difficult to find pointers on how to do
easily and well.

Would you have someplace that you could point me toward so that I could figure
out how to do it easily ?
(Am hoping to work in python. Just not sure what to call what I'm trying to do
- - - coming up with syslog from the python libraries but that's not really what
I'm working on - - - - sensors here there and everywhere and wanting to log
the data (will be doing some use of the data immediately but it is important
that I collect the data).)

TIA

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAPpdf5_g2kCJLZ0egpy_b589MuHefoRaHVshj6K9aeCOBTZpew%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAE5VhgV_fCo93BoRhTCzLKHuuwd-0amciiJ-ophisRwSQH0kLw%40mail.gmail.com.