I'm making an application using Python to parse a webpage. For reference, this is the program's code:
import csv
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import logging
logging.basicConfig(filename='example.log',level=logging.DEBUG)
logging.info('first log')
listsof = []
driver = webdriver.Firefox()
logging.info('Setup driver')
driver.implicitly_wait(30)
logging.info('next step')
driver.get("https://www.lacrossealerts.com/login")
logging.info('Got webpage')
username = driver.find_element_by_name('username')
password = driver.find_element_by_name('password')
login = driver.find_element_by_name('login')
logging.info('Got elements')
username.send_keys("username")
password.send_keys("password")
password.send_keys(Keys.RETURN)
logging.info('Hit enter')
test = driver.find_elements_by_class_name('row-val')
contents = driver.page_source
test1 = driver.find_elements_by_class_name('row-val')
test2 = driver.find_element_by_class_name('timestamp')
logging.info('Found elements')
listsof.append(test2.text)
for e in test1:
print(e.text)
listsof.append(e.text)
print(listsof)
logging.info('Step before write to CSV')
with open('C:/Users/MyUser/Downloads/weather.csv', 'a', newline='') as csvfile:
writer = csv.writer(csvfile, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(listsof)
driver.close()
logging.info('Wrote to CSV')
In this, I had some logging. I get this:
INFO:root:first log
DEBUG:selenium.webdriver.remote.remote_connection:POST http://127.0.0.1:65058/hub/session {"sessionId": null, "desiredCapabilities": {"platform": "ANY", "javascriptEnabled": true, "version": "", "browserName": "firefox"}}
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request
INFO:root:Setup driver
DEBUG:selenium.webdriver.remote.remote_connection:POST http://127.0.0.1:65058/hub/session/5cdbbf60-3a9f-4eca-96d2-7a026c47b56d/timeouts/implicit_wait {"sessionId": "5cdbbf60-3a9f-4eca-96d2-7a026c47b56d", "ms": 30000.0}
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request
INFO:root:next step
DEBUG:selenium.webdriver.remote.remote_connection:POST http://127.0.0.1:65058/hub/session/5cdbbf60-3a9f-4eca-96d2-7a026c47b56d/url {"url": "https://www.lacrossealerts.com/login", "sessionId": "5cdbbf60-3a9f-4eca-96d2-7a026c47b56d"}
DEBUG:selenium.webdriver.remote.remote_connection:Finished Request
INFO:root:Got webpage
DEBUG:selenium.webdriver.remote.remote_connection:POST http://127.0.0.1:65058/hub/session/5cdbbf60-3a9f-4eca-96d2-7a026c47b56d/element {"using": "name", "sessionId": "5cdbbf60-3a9f-4eca-96d2-7a026c47b56d", "value": "username"}
So it looks like it doesn't get past one of these lines:
username = driver.find_element_by_name('username')
password = driver.find_element_by_name('password')
login = driver.find_element_by_name('login')
Could this be a network or internet problem, that it doesn't have the proper permissions? Is there something else? When I run this in a batch file:
cd C:\Users\User\Downloads
python weather.py
It works perfectly. I'm running this as a scheduled task, on my user account. When it is set to Run only when this user is logged on
, it works perfectly. However, when I try to run it on the System user account or Run whether this user is logged on or not
, it doesn't complete the task. I would like for the task (cmd and launches Firefox) to not be displayed to the user, as to not disturb their work, since it is running every 20 minutes, but this isn't working. Is there a problem with the way I'm doing it? Is there a better way to "hide" this from the user?
try-except
, since I'm not sure what exception to occur. Any thoughts?