Flink 1.5.1 on yarn

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP



Flink 1.5.1 on yarn



I'm using flink 1.5.1 , that one cluster for one job



I submit a job to yarn wiht CLI, and then to cancel it ,there's sonething wrong that I can't understant


:~/flink-1.5.1/bin$ ./flink run -d -m yarn-cluster -ynm lz_test_alone1 -ytm 4096 -yjm 1024 -c Demo /home/test/flink1.5_demo-1.1-SNAPSHOT.jar
2018-08-10 15:05:28,390 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-sloth.
2018-08-10 15:05:28,390 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-sloth.
2018-08-10 15:05:28,937 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2018-08-10 15:05:28,937 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2018-08-10 15:05:29,324 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Cluster specification: ClusterSpecificationmasterMemoryMB=1024, taskManagerMemoryMB=4096, numberTaskManagers=1, slotsPerTaskManager=1
2018-08-10 15:05:29,818 WARN org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory - The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
2018-08-10 15:05:29,824 WARN org.apache.flink.yarn.AbstractYarnClusterDescriptor - The configuration directory ('/home/flink-1.5.1/conf') contains both LOG4J and Logback configuration files. Please delete or rename one of them.
2018-08-10 15:05:31,087 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Submitting application master application_1526888270443_0071
2018-08-10 15:05:31,121 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1526888270443_0071
2018-08-10 15:05:31,121 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Waiting for the cluster to be allocated
2018-08-10 15:05:31,126 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying cluster, current state ACCEPTED
2018-08-10 15:05:35,644 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN application has been deployed successfully.
2018-08-10 15:05:35,644 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - The Flink YARN client has been started in detached mode. In order to stop Flink on YARN, use the following command or a YARN web interface to stop it:
yarn application -kill application_1526888270443_0071
Please also note that the temporary files of the YARN session in the home directory will not be removed.
Job has been submitted with JobID 4c4f9564adc94cd38d91d46660ade0d9

:~/flink-1.5.1/bin$ ./flink cancel 4c4f9564adc94cd38d91d46660ade0d9
2018-08-10 15:05:55,484 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-sloth.
2018-08-10 15:05:55,484 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-sloth.
Cancelling job 4c4f9564adc94cd38d91d46660ade0d9.
2018-08-10 15:05:55,837 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - YARN properties set default parallelism to 3
2018-08-10 15:05:55,837 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - YARN properties set default parallelism to 3
YARN properties set default parallelism to 3
2018-08-10 15:05:56,026 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2018-08-10 15:05:56,026 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2018-08-10 15:05:56,116 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Found application JobManager host name 'cluster23.org' and port '57813' from supplied application id 'application_1526888270443_0067'
2018-08-10 15:05:56,707 WARN org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory - The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.



when I use 'flink run' that return me the jobID and applicationId (application_1526888270443_0071);



thenI use 'flink cancel ' and the log show another applicationId (application_1526888270443_0067), so it goes wrong .



Is there any one can help me?




1 Answer
1


./flink cancel



use on yarn job need -yid
this srcipt should be ：


./flink cancel 4c4f9564adc94cd38d91d46660ade0d9 -yid application_1526888270443_0071



this works






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

-

Popular posts from this blog

make 2 or more post in bootsrap

Image
Clash Royale CLAN TAG #URR8PPP make 2 or more post in bootsrap I am making a CMS and I want to add two or more content post in every row, I can made one post in one columns, please see my code below. I am using PHP , HTML and bootstrap . CMS PHP HTML bootstrap How can I add two or more content post in every row? Code: <div class="container"> <div class="row"> <!-- Post Content Column --> <div class="col-lg-4 border border-dark rounded"> <?php $query = "SELECT * FROM POSTS"; $run_query_post = mysqli_query($connection, $query); while ($row = mysqli_fetch_assoc($run_query_post)) $post_id = $row['post_id']; $post_title = $row['post_title']; $post_author = $row['post_author']; $post_date = $row['post_date']; $post_image = $row['post_image']; $post_content = substr($row['post_content'], 0, 50); $post_status = $row['post_status']; if ($post_status !=...
Read more

Store custom data using WC_Cart add_to_cart() method in Woocommerce 3

Image
Clash Royale CLAN TAG #URR8PPP Store custom data using WC_Cart add_to_cart() method in Woocommerce 3 I am creating a membership site and totally created static pages for each Membership plans (have only 3 plans). However, I have added products for each plan and when I hit SELECT PLAN button I redirect to some custom form where I ask users range of info we are going to use to fulfil the plan (same as sneakertub.com). I have written code into the PHP page which will handle SUBMIT action of the form. This PHP file, infopage.php , will process POST data I sent via POST call and stores these all data into WC session. infopage.php $customer_name = $_POST["customer_name"]; $customer_email = $_POST["customer_email"]; $customer_sex = $_POST["customer_sex"]; $customer_age = $_POST["customer_age"]; $product_id = $_POST["product_id"]; global $wp_session; $data = array( 'customer_name' => $customer_name, 'customer_email' =...
Read more

Firebase Auth - with Email and Password - Check user already registered

Image
Clash Royale CLAN TAG #URR8PPP Firebase Auth - with Email and Password - Check user already registered I want to check when a user attempts to signup with createUserWithEmailAndPassword() in Firebase user Authentication method, this user is already registered with my app. createUserWithEmailAndPassword() 8 Answers 8 To detect whether a user with that email address already exists, you can detect when the call to createUserWithEmailAndPassword () fails with auth/email-already-in-use . I see that @Srinivasan just posted an answer for this. createUserWithEmailAndPassword () auth/email-already-in-use Alternatively, you can detect that an email address is already used by calling fetchSignInMethodsForEmail() (previous called fetchProvidersForEmail() ). fetchSignInMethodsForEmail() fetchProvidersForEmail() When the user trying to create an user with same email address, the task response will be "Response: The email address is already in use by another account." mFirebas...
Read more