unable to scrape the current stock price from yahoo finance website using scrapy

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP



unable to scrape the current stock price from yahoo finance website using scrapy



I want to scrape the Current Stock Value of any company using scrapy whenever I run the spider from the below Yahoo Finance :


scrapy


spider



But i am unable to extract it using scrapy shell as shown in screenshots.



Screenshot of yahoo finance with red rectangles(details need to be scraped) :








1 Answer
1



If you can't find an element that is supposed to be in a page, I suggest you use view(response). It shows you how the scrapy see the page.
In this case, if you use it by yourself, you can see that in fetched HTML, there is no price element. I set a normal user agent in the request and it worked properly


view(response)


request=scrapy.Request('https://in.finance.yahoo.com/quote/ZUO/key-statistics?p=ZUO', headers='User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0')
fetch(request)
response.xpath('//*[@id="quote-header-info"]/div[3]/div/div/span[1]').extract()



Result:


[u'<span class="Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)" data-reactid="35">27.40</span>']






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

-

Popular posts from this blog

make 2 or more post in bootsrap

Image
Clash Royale CLAN TAG #URR8PPP make 2 or more post in bootsrap I am making a CMS and I want to add two or more content post in every row, I can made one post in one columns, please see my code below. I am using PHP , HTML and bootstrap . CMS PHP HTML bootstrap How can I add two or more content post in every row? Code: <div class="container"> <div class="row"> <!-- Post Content Column --> <div class="col-lg-4 border border-dark rounded"> <?php $query = "SELECT * FROM POSTS"; $run_query_post = mysqli_query($connection, $query); while ($row = mysqli_fetch_assoc($run_query_post)) $post_id = $row['post_id']; $post_title = $row['post_title']; $post_author = $row['post_author']; $post_date = $row['post_date']; $post_image = $row['post_image']; $post_content = substr($row['post_content'], 0, 50); $post_status = $row['post_status']; if ($post_status !=...
Read more

Store custom data using WC_Cart add_to_cart() method in Woocommerce 3

Image
Clash Royale CLAN TAG #URR8PPP Store custom data using WC_Cart add_to_cart() method in Woocommerce 3 I am creating a membership site and totally created static pages for each Membership plans (have only 3 plans). However, I have added products for each plan and when I hit SELECT PLAN button I redirect to some custom form where I ask users range of info we are going to use to fulfil the plan (same as sneakertub.com). I have written code into the PHP page which will handle SUBMIT action of the form. This PHP file, infopage.php , will process POST data I sent via POST call and stores these all data into WC session. infopage.php $customer_name = $_POST["customer_name"]; $customer_email = $_POST["customer_email"]; $customer_sex = $_POST["customer_sex"]; $customer_age = $_POST["customer_age"]; $product_id = $_POST["product_id"]; global $wp_session; $data = array( 'customer_name' => $customer_name, 'customer_email' =...
Read more

Firebase Auth - with Email and Password - Check user already registered

Image
Clash Royale CLAN TAG #URR8PPP Firebase Auth - with Email and Password - Check user already registered I want to check when a user attempts to signup with createUserWithEmailAndPassword() in Firebase user Authentication method, this user is already registered with my app. createUserWithEmailAndPassword() 8 Answers 8 To detect whether a user with that email address already exists, you can detect when the call to createUserWithEmailAndPassword () fails with auth/email-already-in-use . I see that @Srinivasan just posted an answer for this. createUserWithEmailAndPassword () auth/email-already-in-use Alternatively, you can detect that an email address is already used by calling fetchSignInMethodsForEmail() (previous called fetchProvidersForEmail() ). fetchSignInMethodsForEmail() fetchProvidersForEmail() When the user trying to create an user with same email address, the task response will be "Response: The email address is already in use by another account." mFirebas...
Read more