Re: parse xml from url to database

quarta-feira, 12 de Junho de 2019 às 17:22:01 UTC+1, Nuno Vieira escreveu:

Hi there.

I´m having some problems on parsing a real estate xml to database.

The user profile have url field to import properties automatically from xml feed (url). The user can insert mannually or automatically, if the user insert the url feed then it should be able to get the objects and save them to mysql database automatically.

The function is not working and i do not know why, gives no error but don´t populate the database.

Models.py:

class Listing(models.Model):

'''Create table listing in db '''

realtor = models.ForeignKey(User, on_delete=models.CASCADE)

title = models.CharField(max_length=200)

address = models.CharField(max_length=200)

city = models.CharField(max_length=100)

state = models.CharField(max_length=100)

zipcode = models.CharField(max_length=20)

longitude = models.DecimalField(max_digits=9, decimal_places=6,default=0)

latitude = models.DecimalField(max_digits=9, decimal_places=6, default=0)

description = models.TextField(blank=True)

price = models.IntegerField()

bedrooms = models.IntegerField()

bathrooms = models.DecimalField(max_digits=2, decimal_places=1)

garage = models.IntegerField(default=0)

sqmt = models.IntegerField(blank=True)

lot_size = models.DecimalField(max_digits=5, decimal_places=1,blank=True)

On_Processing = 'On Processing'

AMais = 'A+'

A = 'A'

B = 'B'

BMenos = 'B-'

C = 'C'

D = 'D'

E = 'E'

F = 'F'

ENERGY_RATE_CHOICES = [

(On_Processing, 'On Processing'),

(AMais, 'A+'),

(A, 'A'),

(B, 'B'),

(BMenos, 'B-'),

(C, 'C'),

(D, 'D'),

(E, 'E'),

(F, 'F'),

]

energy_rate = models.CharField(

max_length=12,

choices=ENERGY_RATE_CHOICES,

default=On_Processing,

)

photo_main = models.ImageField(upload_to='photos/%Y/%m/%d/', default='default.jpg')

photo_main_url = models.URLField(blank = True)

is_published = models.BooleanField(default=True)

list_date = models.DateTimeField(default=datetime.now, blank=True)

video_url = models.CharField(max_length=100, blank=True)

virtual_tour = models.URLField(blank=True)

visit_num = models.PositiveIntegerField(default=0)

def __str__(self):

return self.title

def get_absolute_url(self):

return reverse('listing-detail', kwargs={'pk': self.pk})

Function on xml_parse.py:

...

import xml.etree.ElementTree as ET

import requests

from users.models import Profile

def xml_parse(self):

prof_list = Profile.objects.all()

for prof in prof_list:

if prof.xml_ky is not None:

url = xml_ky

response = request.get(url)

with open('xml/%s' % self.id + '/feed.xml', 'wb') as file:

file.write(response.content)

tree = ET.parse('xml/%s' % self.id + '/feed.xml')

root = tree.getroot()

for props in root.iter('property'):

listing, created = Listing.object.create(

title = props.find('type').text + ' ' + props.find('town').text,

address = props.find('location_detail').text,

city = props.find('town').text,

state = props.find('province').text,

zipcode = props.find('zipcode').text,

loc = props.find('location'),

longitude = loc.find('longitude').text,

latitude = loc.find('latitude').text,

desc = props.find('desc'),

description = desc.find('en').text,

price = props.find('price').text,

bedrooms = props.find('beds').text,

bathrooms = props.find('baths').text,

garage = props.find('garage').text,

area = props.find('surface_area'),

sqmt = area.find('built').text,

energy_rate = props.find('energy_rate').text,

realtor = self.request.user,

img = props.find('images'),

image = img.get('id=1'),

image_url = image.find('url').text,

photo_main_url = image_url,

video_url = props.find('video_url').text,

virtual_tour = props.find('virtual_tour').text,

)

if created:

return listing.save()

If you could help me and point me in the right direction it would be great.

Thanks and sorry for the black background!

Thanks for the reply.

 XML Example:

<?xml version="1.0" encoding="UTF-8"?>

<root>

  <kyero>

    <feed_version>3</feed_version>

  </kyero>

<property>

<id>2150</id>

<date>2018-06-27 11:18:37</date>

<ref>BC10002</ref>

<price>1599900</price>

<currency>EUR</currency>

<price_freq>sale</price_freq>

<part_ownership>0</part_ownership>

<leasehold>0</leasehold>

<new_build>0</new_build>

<surface_area>

<built>6000</built>

</surface_area>

<type>Villa</type>

<location>

<latitude>37.134720</latitude>

<longitude>-8.020436</longitude>

</location>

<location_detail>optional location detail</location_detail>

<town>Loulé</town>

<province>Algarve</province>

<country>Portugal</country>

<beds>5</beds>

<baths>5</baths>

<pool>1</pool>

<images>

<image id="1">

<url>http://bc-villas.com/wp-content/uploads/2018/05/001-DJI_0243.jpg</url>

</image>

<image id="2">

<url>http://bc-villas.com/wp-content/uploads/2018/05/001b-IMG_3891.jpg</url>

</image>

<image id="3">

<url>http://bc-villas.com/wp-content/uploads/2018/05/001c-IMG_3878.jpg</url>

</image>

<image id="4">

<url>http://bc-villas.com/wp-content/uploads/2018/05/002-DJI_0233.jpg</url>

</image>

<image id="5">

<url>http://bc-villas.com/wp-content/uploads/2018/05/003-DJI_0221.jpg</url>

</image>

</images>

<features>

<feature/>

</features>

<url>

<en>http://bc-villas.com/property/private-modern-5-bedrooms-villa/</en>

</url>

<desc>

<en>This truly stunning  Private modern 5 bedrooms villa.

Set in 6,000m2 accessed by two security gates.

Located just below the old market town of Loule and only 8 minute drive to Vilasol and Vilamoura and everything you will need for luxury living .

10 minutes drive of the villa you will find championship golf courses, numerous restaurants, golden beaches, sports and leisure facilities. The famous water park is within 5 min drive .

Underfloor heating

Surround sound all rooms

Central vac

Solor power

Outside kitchen

Sold fully furnished

Brand new borehole

Alarm

Intercom

Central shutters full lock down

Pool

Water softer

Fast internet</en>

</desc>

</property>

<property>

<id>2777</id>

<date>2018-06-20 14:12:13</date>

<ref>BC10004</ref>

<price>690000</price>

<currency>EUR</currency>

<price_freq>sale</price_freq>

<part_ownership>0</part_ownership>

<leasehold>0</leasehold>

<new_build>0</new_build>

<surface_area>

<built>250</built>

</surface_area>

<type>Villa</type>

<location>

<latitude>37.134720</latitude>

<longitude>-8.020436</longitude>

</location>

<location_detail>optional location detail</location_detail>

<town>São Brás de Alportel</town>

<province>Algarve</province>

<country>Portugal</country>

<beds>4</beds>

<baths>4</baths>

<pool>1</pool>

<images>

<image id="1">

<url>http://bc-villas.com/wp-content/uploads/2018/06/DJI_0162-1.jpg</url>

</image>

<image id="2">

<url>http://bc-villas.com/wp-content/uploads/2018/06/DJI_0163-1.jpg</url>

</image>

<image id="3">

<url>http://bc-villas.com/wp-content/uploads/2018/06/DJI_0166-1.jpg</url>

</image>

<image id="4">

<url>http://bc-villas.com/wp-content/uploads/2018/06/DJI_0169.jpg</url>

</image>

<image id="5">

<url>http://bc-villas.com/wp-content/uploads/2018/06/IMG_4073-1.jpg</url>

</image>

</images>

<features>

<feature/>

</features>

<url>

<en>http://bc-villas.com/property/villa-v4-sao-bras-de-alportel/</en>

</url>

<desc>

<en>Villa situated in São Brás de Alportel, 20 minutes from the beach and Faro airport.

Isolated, with pool and alarm, 4 bedrooms (2 en suite). A privileged view over the countryside and town. A place for lovers of nature and tranquility, where family days are very well enjoyed both outdoors and in the games room that the house has. An excellent investment both for fixed address and for holidays.

</en>

</desc>

</property>

<property>

<id>2779</id>

<date>2018-06-20 14:15:54</date>

<ref>BC10001</ref>

<price>649000</price>

<currency>EUR</currency>

<price_freq>sale</price_freq>

<part_ownership>0</part_ownership>

<leasehold>0</leasehold>

<new_build>0</new_build>

<surface_area>

<built>2924</built>

</surface_area>

<type>Villa</type>

<location>

<latitude>37.134720</latitude>

<longitude>-8.020436</longitude>

</location>

<location_detail>optional location detail</location_detail>

<town>São Brás de Alportel</town>

<province>Algarve</province>

<country>Portugal</country>

<beds>4</beds>

<baths>3</baths>

<pool>1</pool>

<images>

<image id="1">

<url>http://bc-villas.com/wp-content/uploads/2018/06/018-noite1.jpg</url>

</image>

<image id="2">

<url>http://bc-villas.com/wp-content/uploads/2018/06/019-noite2.jpg</url>

</image>

<image id="3">

<url>http://bc-villas.com/wp-content/uploads/2018/06/020-noite3.jpg</url>

</image>

<image id="4">

<url>http://bc-villas.com/wp-content/uploads/2018/06/020-noite4.jpg</url>

</image>

<image id="5">

<url>http://bc-villas.com/wp-content/uploads/2018/06/barbecue1.jpg</url>

</image>

</images>

<features>

<feature/>

</features>

<url>

<en>http://bc-villas.com/property/family-villa-in-sao-bras-de-alportel/</en>

</url>

<desc>

<en>A perfect villa for a family, located in São Brás de Alportel, a few kilometers from the beach, the airport and the city center. In a place where you can enjoy the tranquility and nature. It has excellent finishings, where we would like to highlight the skylight in the living room, allowing the sun light to go inside directly from the roof.

It has swimming pool, 4 bedrooms (one of them as a suite), 3 toilets in total, 2 covered garages, two entrances for cars with automatic gates, barbecue, alarm, air-conditioning, stove and many other functionalities.

A fantastic opportunity not to be missed!</en>

</desc>

</property>

</root>

 

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/6e59e9c1-84bb-49df-8ea1-bd6be3f097f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.