Add the support of ANALYZE command to inspect the performance of RocksDB

[chrisxu333]

This pr tries to close #783 .

To expose essential perf_context and iostats_context from rocksdb, a new command ANALYZE is introduced.

A typical usage would be as follows:

[mapleFU]

This is a bit confusing because it does far less than PROFILE in SQL. Also it might break the previous config about perflevel, so I wonder whether we have better way for this...

[mapleFU]

Maybe it's better to first having a RAII guard like TiKV: https://github.com/tikv/tikv/blob/master/src/coprocessor/tracker.rs ( We can be easier than them) ?

[chrisxu333]

Maybe it's better to first having a RAII guard like TiKV: https://github.com/tikv/tikv/blob/master/src/coprocessor/tracker.rs ( We can be easier than them) ?

Can you elaborate more on this? I didn't quite get the point.

[mapleFU]

Maybe it's better to first having a RAII guard like TiKV: https://github.com/tikv/tikv/blob/master/src/coprocessor/tracker.rs ( We can be easier than them) ?

Can you elaborate more on this? I didn't quite get the point.

Hmm we can just leave it here in this patch. I believe something like a :

class StoragePerfContext {
  
  void close();
  ~StoragePerfContext() {
      if (!reset_) {
         rocksdb::Perfcontext // resume
      }
   }
private:
  Storage* // or rocksdb?
  bool reseted_ = false;
};

Might be better? If other command want to debugging, they can also uses this tool? Also we can considering RAII here.

[chrisxu333]

Maybe it's better to first having a RAII guard like TiKV: https://github.com/tikv/tikv/blob/master/src/coprocessor/tracker.rs ( We can be easier than them) ?

Can you elaborate more on this? I didn't quite get the point.

Hmm we can just leave it here in this patch. I believe something like a :

class StoragePerfContext {
  
  void close();
  ~StoragePerfContext() {
      if (!reset_) {
         rocksdb::Perfcontext // resume
      }
   }
private:
  Storage* // or rocksdb?
  bool reseted_ = false;
};

Might be better? If other command want to debugging, they can also uses this tool? Also we can considering RAII here.

I see what you meant. So just to be clear, do we still need to do this in this patch? Or we wait until when it's actually needed to do so?

[mapleFU]

do we still need to do this in this patch

I agree it's not neccessary in this patch. Personally I think metrics can be widely used, but lets first get it in used...

cc @PragmaTwice @git-hulk This command looks like SQL "PROFILE" with rocksdb io/perf stats enabled and output. It can be used in some cases, but for some blocking command I don't know whether it's ok. Would you mind take a look?

[chrisxu333]

do we still need to do this in this patch

I agree it's not neccessary in this patch. Personally I think metrics can be widely used, but lets first get it in used...

cc @PragmaTwice @git-hulk This command looks like SQL "PROFILE" with rocksdb io/perf stats enabled and output. It can be used in some cases, but for some blocking command I don't know whether it's ok. Would you mind take a look?

@git-hulk @PragmaTwice Hello guys, if you have time could you help review this pr? Thank you :)

[git-hulk]

Sure, sorry for missing this message. I will take a look recently.

[git-hulk]

@chrisxu333 Can rename true_args_ to command_args_, rest are good to me. Sorry for missing this point.

[mapleFU]

Would you mind fix the lint first?

[torwig]

@chrisxu333 ./x.py format should do the trick.

[chrisxu333]

Would you mind fix the lint first?

Sounds good I just did.

[mapleFU]

Should we check the arity before Parse? Since wrong command_args_ might make our program crash

[PragmaTwice]

We need to reuse the code in ExecuteCommand rather than write them again here.

[chrisxu333]

I just pushed a newer version which added the arity checking logic. As for reusing / refactoring the code in ExecuteCommand, maybe it's better to do it in a separate pr? As the ExecuteCommand code needs to be dissected into several chunks for reusing purpose. @mapleFU @PragmaTwice

[mapleFU]

nit: we can directly insert rather than push back

[chrisxu333]

@chrisxu333 ./x.py format should do the trick.

Hi @torwig, could you help review this pr whenever you're able to? Thx!

[chrisxu333]

Hi @git-hulk @PragmaTwice,

Could you help review and approve this pr if possible? Thx!

[git-hulk]

@chrisxu333 Thanks for your follow-up. This PR looks good to me, to see if @PragmaTwice and @torwig has any comments, if not, we can move forward to merge.

[sonarcloud]

Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

1 New issue
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud