2019 Q2 Hyperledger Caliper

Created by qinghui hou, last modified by Raghunathan P on Jan 11, 2022

Project Health

Caliper continues to grow and mature.In the past quarter, one of many contributors has grown the maintainer according to his great efforts. There were still many consulting questions in rocket chat and issues,most of which were well answered.Besides that, Caliper intern also participated in the project development and is in charge of Caliper visualization.

Issues

There are no major issues at this time.

Releases

Caliper npm packages and fixed API is on the way. After that, Caliper plans to release alpha version.

Overall Activity in the Past Quarter

During the past Quarter, 30 issues have been closed, 41 PRs have been reviewed and merged. Now the project has 290+ stars and 180+ forks.Thanks every contributor’s efforts. Below are some main updates in the past 3 months:

  • Support the latest iroha;
  • Support the Ethereum benchmark;
  • Refactored Caliper code,split Caliper code base into modules.
  • Created CLI and sample package.
  • Added Fabric kafka example and raft example;
  • Enhanced abilities, such as automatically generate crypto config, a license check process and Caliper running environment check.
  • Improved documentation, e.g.Ethereum adaptor configuration doc,docs for Caliper CLI and runtime configuration doc. 

Current Plans

Real time charts

high

Various reports generation

medium

Integrate with Grafana

High

Metrics extension

Low

Docker image

Medium

Maintainer Diversity

Now the maintainers are from Huawei,Budapest University and IBM.

  • Attila Klenik ( Budapest University )  
  • Nick Lincoln (IBM)
  • Yu Pan (Huawei)
  • Feihu Jiang (Huawei)

Contributor Diversity

There are 25 contributors from various companies/organisations such as Huawei, IBM, Intel, Soramitsu, Budapest University, Monax and the university of Oregon. And we are connecting with several companies now to try to attract more contributors.

Additional Information

N/A

Reviewed by

  • ✅ Arnaud Le Hors
  • ✅ Baohua Yang
  • ✅ Binh Nguyen
  • ✅ Christopher Ferris
  • ✅ Dan Middleton
  • ✅ Hart Montgomery
  • ✅ Kelly Olson
  • ✅ Mark Wagner
  • ✅ Mic Bowman
  • ✅ Nathan George
  • 🔲 Silas Davis

Comments:

Do you feel that Caliper is aligned with the PSWG whitepaper in terms of metric definitions and reporting?

Posted by dan.middleton@intel.com at Jun 20, 2019 14:30

Hi Dan! Yes, the TX data gathered by Caliper is sufficient to produce the high-level client-side (end-to-end) metrics defined by the whitepaper. The generated report will be refined in the coming weeks, the Caliper UI internship project will shed some light on how to present the data in the best way (and to select the interesting views/visuals/numbers).

Posted by klenik at Jun 25, 2019 09:18

That sounds good. I'm particularly interested that the reporting reflects network size, which was one of the key definitions of the whitepaper.

As you mention UI, please consider incorporating accessibility requirements into the UI if those requirements don't already exist.

Posted by dan.middleton@intel.com at Jul 01, 2019 13:53

Well, network size is a tricky question. Caliper doesn't necessarily need to know about the entire network (especially after we add the discovery feature for Fabric, for example). So we can't really "infer the truth" about the whole network topology from client-side, we rely on the user-provided settings.

But users can include arbitrary metadata about the test in the test configuration, and those metadata are included in the report. This is the current way to specify the network size in the tests. But we'll think about how to automatically derive the (know) network topology and include it in the report.

Can you elaborate on the accessibility requirements? Do you mean simple authentication features, like for a grafana server? Or more exotic data access control, like with Hyperledger Explorer?  

Posted by klenik at Jul 10, 2019 08:20

Hi i am trying to implement a blockchain application for my research. Where can I find step by step tutorial to integrate Caliper with Ethereum for benchmarking the implementation?

Posted by raghunathanp95 at Jan 11, 2022 20:15

Raghunathan P – You can find some tutorials for Caliper at: https://www.hyperledger.org/use/tutorials

And for additional questions about Caliper, I recommend asking on the Caliper chat channels, mailing list or in our of their community calls.  Links to those are at: Hyperledger Caliper

Posted by davidwboswell at Jan 11, 2022 22:22