Skip to content
This repository was archived by the owner on Nov 6, 2020. It is now read-only.

Commit 7bf16cc

Browse files
andresilva5chdn
authored andcommitted
[beta] Backports (#8785)
* Fix light sync with initial validator-set contract (#8528) * Fix #8468 * Use U256::max_value() instead * Fix again * Also change initial transaction gas * Resumable warp-sync / Seed downloaded snapshots (#8544) * Start dividing sync chain : first supplier method * WIP - updated chain sync supplier * Finish refactoring the Chain Sync Supplier * Create Chain Sync Requester * Add Propagator for Chain Sync * Add the Chain Sync Handler * Move tests from mod -> handler * Move tests to propagator * Refactor SyncRequester arguments * Refactoring peer fork header handler * Fix wrong highest block number in snapshot sync * Small refactor... * Resume warp-sync downloaded chunks * Add comments * Refactoring the previous chunks import * Fix tests * Address PR grumbles * Fix not seeding current snapshot * Address PR Grumbles * Address PR grumble * Retry failed CI job * Update SnapshotService readiness check Fix restoration locking issue for previous chunks restoration * Fix tests * Fix tests * Fix test * Early abort importing previous chunks * PR Grumbles * Update Gitlab CI config * SyncState back to Waiting when Manifest peers disconnect * Move fix * Better fix * Revert GitLab CI changes * Fix Warning * Refactor resuming snapshots * Fix string construction * Revert "Refactor resuming snapshots" This reverts commit 75fd4b5. * Update informant log * Fix string construction * Refactor resuming snapshots * Fix informant * PR Grumbles * Update informant message : show chunks done * PR Grumbles * Fix * Fix Warning * PR Grumbles * Don't open Browser post-install on Mac (#8641) Since we start parity with the UI disabled per default now, opening the browser post installation will show an annoying error message, confusing the user. This patch removes opening the browser to prevent that annoyance. fixes #8194 * Fix not downloading old blocks (#8642) * Fix PoW blockchains sealing notifications in chain_new_blocks (#8656) * Shutdown the Snapshot Service early (#8658) * Shutdown the Snapshot Service when shutting down the runner * Rename `service` to `client_service` * Fix tests * Fix cli signer (#8682) * Update ethereum-types so `{:#x}` applies 0x prefix * Set the request index to that of the current request (#8683) * Set the request index to that of the current request When setting up the chain of (two) requests to look up a block by hash, the second need to refer to the first. This fixes an issue where the back ref was set to the subsequent request, not the current one. When the requests are executed we loop through them in order and ensure the requests that should produce headers all match up. We do this by index so they better be right. In other words: off by one. * network-devp2p: handle UselessPeer disconnect (#8686) * Fix local transactions policy. (#8691) * CI: Fixes for Android Pipeline (#8745) * ci: Remove check for shared libraries in gitlab script * ci: allow android arm build to fail * Custom Error Messages on ENFILE and EMFILE IO Errors (#8744) * Custom Error Messages on ENFILE and EMFILE IO Errors Add custom mapping of ENFILE and EMFILE IO Errors (Failure because of missing system resource) right when chaining ioError into ::util::Network::Error to improve Error Messages given to user Note: Adds libc as a dependency to util/network * Use assert-matches for more readable tests * Fix Wording and consistency * ethcore-sync: fix connection to peers behind chain fork block (#8710)
1 parent 69d6d82 commit 7bf16cc

File tree

30 files changed

+705
-264
lines changed

30 files changed

+705
-264
lines changed

.gitlab-ci.yml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -180,6 +180,23 @@ windows:
180180
paths:
181181
- parity.zip
182182
name: "x86_64-pc-windows-msvc_parity"
183+
android-armv7:
184+
stage: build
185+
image: parity/parity-android:latest
186+
only:
187+
- beta
188+
- tags
189+
- stable
190+
- triggers
191+
script:
192+
- cargo build --target=armv7-linux-androideabi
193+
tags:
194+
- rust-arm
195+
allow_failure: true
196+
artifacts:
197+
paths:
198+
- parity.zip
199+
name: "armv7-linux-androideabi_parity"
183200
docker-build:
184201
stage: build
185202
only:

Cargo.lock

Lines changed: 98 additions & 97 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

ethcore/service/src/service.rs

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ use sync::PrivateTxHandler;
2929
use ethcore::client::{Client, ClientConfig, ChainNotify, ClientIoMessage};
3030
use ethcore::miner::Miner;
3131
use ethcore::snapshot::service::{Service as SnapshotService, ServiceParams as SnapServiceParams};
32-
use ethcore::snapshot::{RestorationStatus};
32+
use ethcore::snapshot::{SnapshotService as _SnapshotService, RestorationStatus};
3333
use ethcore::spec::Spec;
3434
use ethcore::account_provider::AccountProvider;
3535

@@ -168,6 +168,11 @@ impl ClientService {
168168

169169
/// Get a handle to the database.
170170
pub fn db(&self) -> Arc<KeyValueDB> { self.database.clone() }
171+
172+
/// Shutdown the Client Service
173+
pub fn shutdown(&self) {
174+
self.snapshot.shutdown();
175+
}
171176
}
172177

173178
/// IO interface for the Client handler

ethcore/src/client/client.rs

Lines changed: 40 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ use std::collections::{HashSet, HashMap, BTreeMap, BTreeSet, VecDeque};
1818
use std::fmt;
1919
use std::str::FromStr;
2020
use std::sync::{Arc, Weak};
21-
use std::sync::atomic::{AtomicUsize, AtomicBool, Ordering as AtomicOrdering};
21+
use std::sync::atomic::{AtomicBool, Ordering as AtomicOrdering};
2222
use std::time::{Instant};
2323

2424
// util
@@ -208,6 +208,8 @@ pub struct Client {
208208
queue_transactions: IoChannelQueue,
209209
/// Ancient blocks import queue
210210
queue_ancient_blocks: IoChannelQueue,
211+
/// Hashes of pending ancient block wainting to be included
212+
pending_ancient_blocks: RwLock<HashSet<H256>>,
211213
/// Consensus messages import queue
212214
queue_consensus_message: IoChannelQueue,
213215

@@ -461,6 +463,7 @@ impl Importer {
461463
let hash = header.hash();
462464
let _import_lock = self.import_lock.lock();
463465

466+
trace!(target: "client", "Trying to import old block #{}", header.number());
464467
{
465468
trace_time!("import_old_block");
466469
// verify the block, passing the chain for updating the epoch verifier.
@@ -741,6 +744,7 @@ impl Client {
741744
notify: RwLock::new(Vec::new()),
742745
queue_transactions: IoChannelQueue::new(MAX_TX_QUEUE_SIZE),
743746
queue_ancient_blocks: IoChannelQueue::new(MAX_ANCIENT_BLOCKS_QUEUE_SIZE),
747+
pending_ancient_blocks: RwLock::new(HashSet::new()),
744748
queue_consensus_message: IoChannelQueue::new(usize::max_value()),
745749
last_hashes: RwLock::new(VecDeque::new()),
746750
factories: factories,
@@ -1972,7 +1976,7 @@ impl BlockChainClient for Client {
19721976
impl IoClient for Client {
19731977
fn queue_transactions(&self, transactions: Vec<Bytes>, peer_id: usize) {
19741978
let len = transactions.len();
1975-
self.queue_transactions.queue(&mut self.io_channel.lock(), len, move |client| {
1979+
self.queue_transactions.queue(&mut self.io_channel.lock(), move |client| {
19761980
trace_time!("import_queued_transactions");
19771981

19781982
let txs: Vec<UnverifiedTransaction> = transactions
@@ -1996,23 +2000,32 @@ impl IoClient for Client {
19962000

19972001
{
19982002
// check block order
1999-
if self.chain.read().is_known(&header.hash()) {
2003+
if self.chain.read().is_known(&hash) {
20002004
bail!(BlockImportErrorKind::Import(ImportErrorKind::AlreadyInChain));
20012005
}
2002-
let status = self.block_status(BlockId::Hash(*header.parent_hash()));
2003-
if status == BlockStatus::Unknown || status == BlockStatus::Pending {
2004-
bail!(BlockImportErrorKind::Block(BlockError::UnknownParent(*header.parent_hash())));
2006+
2007+
let parent_hash = *header.parent_hash();
2008+
let parent_pending = self.pending_ancient_blocks.read().contains(&parent_hash);
2009+
let status = self.block_status(BlockId::Hash(parent_hash));
2010+
if !parent_pending && (status == BlockStatus::Unknown || status == BlockStatus::Pending) {
2011+
bail!(BlockImportErrorKind::Block(BlockError::UnknownParent(parent_hash)));
20052012
}
20062013
}
20072014

2008-
match self.queue_ancient_blocks.queue(&mut self.io_channel.lock(), 1, move |client| {
2009-
client.importer.import_old_block(
2015+
self.pending_ancient_blocks.write().insert(hash);
2016+
2017+
trace!(target: "client", "Queuing old block #{}", header.number());
2018+
match self.queue_ancient_blocks.queue(&mut self.io_channel.lock(), move |client| {
2019+
let result = client.importer.import_old_block(
20102020
&header,
20112021
&block_bytes,
20122022
&receipts_bytes,
20132023
&**client.db.read(),
20142024
&*client.chain.read()
2015-
).map(|_| ()).unwrap_or_else(|e| {
2025+
);
2026+
2027+
client.pending_ancient_blocks.write().remove(&hash);
2028+
result.map(|_| ()).unwrap_or_else(|e| {
20162029
error!(target: "client", "Error importing ancient block: {}", e);
20172030
});
20182031
}) {
@@ -2022,7 +2035,7 @@ impl IoClient for Client {
20222035
}
20232036

20242037
fn queue_consensus_message(&self, message: Bytes) {
2025-
match self.queue_consensus_message.queue(&mut self.io_channel.lock(), 1, move |client| {
2038+
match self.queue_consensus_message.queue(&mut self.io_channel.lock(), move |client| {
20262039
if let Err(e) = client.engine().handle_message(&message) {
20272040
debug!(target: "poa", "Invalid message received: {}", e);
20282041
}
@@ -2131,7 +2144,7 @@ impl ImportSealedBlock for Client {
21312144
route
21322145
};
21332146
let (enacted, retracted) = self.importer.calculate_enacted_retracted(&[route]);
2134-
self.importer.miner.chain_new_blocks(self, &[h.clone()], &[], &enacted, &retracted, true);
2147+
self.importer.miner.chain_new_blocks(self, &[h.clone()], &[], &enacted, &retracted, self.engine.seals_internally().is_some());
21352148
self.notify(|notify| {
21362149
notify.new_blocks(
21372150
vec![h.clone()],
@@ -2433,35 +2446,38 @@ impl fmt::Display for QueueError {
24332446

24342447
/// Queue some items to be processed by IO client.
24352448
struct IoChannelQueue {
2436-
currently_queued: Arc<AtomicUsize>,
2449+
queue: Arc<Mutex<VecDeque<Box<Fn(&Client) + Send>>>>,
24372450
limit: usize,
24382451
}
24392452

24402453
impl IoChannelQueue {
24412454
pub fn new(limit: usize) -> Self {
24422455
IoChannelQueue {
2443-
currently_queued: Default::default(),
2456+
queue: Default::default(),
24442457
limit,
24452458
}
24462459
}
24472460

2448-
pub fn queue<F>(&self, channel: &mut IoChannel<ClientIoMessage>, count: usize, fun: F) -> Result<(), QueueError> where
2449-
F: Fn(&Client) + Send + Sync + 'static,
2461+
pub fn queue<F>(&self, channel: &mut IoChannel<ClientIoMessage>, fun: F) -> Result<(), QueueError>
2462+
where F: Fn(&Client) + Send + Sync + 'static
24502463
{
2451-
let queue_size = self.currently_queued.load(AtomicOrdering::Relaxed);
2452-
ensure!(queue_size < self.limit, QueueError::Full(self.limit));
2464+
{
2465+
let mut queue = self.queue.lock();
2466+
let queue_size = queue.len();
2467+
ensure!(queue_size < self.limit, QueueError::Full(self.limit));
24532468

2454-
let currently_queued = self.currently_queued.clone();
2469+
queue.push_back(Box::new(fun));
2470+
}
2471+
2472+
let queue = self.queue.clone();
24552473
let result = channel.send(ClientIoMessage::execute(move |client| {
2456-
currently_queued.fetch_sub(count, AtomicOrdering::SeqCst);
2457-
fun(client);
2474+
while let Some(fun) = queue.lock().pop_front() {
2475+
fun(client);
2476+
}
24582477
}));
24592478

24602479
match result {
2461-
Ok(_) => {
2462-
self.currently_queued.fetch_add(count, AtomicOrdering::SeqCst);
2463-
Ok(())
2464-
},
2480+
Ok(_) => Ok(()),
24652481
Err(e) => Err(QueueError::Channel(e)),
24662482
}
24672483
}

0 commit comments

Comments
 (0)