diff --git a/.attach_pid13921 b/.attach_pid13921 new file mode 100644 index 000000000..e69de29bb diff --git a/.classpath b/.classpath new file mode 100644 index 000000000..3b5e60bbd --- /dev/null +++ b/.classpath @@ -0,0 +1,14 @@ + + + + + + + + + + + + + + diff --git a/.project b/.project new file mode 100644 index 000000000..136f6680a --- /dev/null +++ b/.project @@ -0,0 +1,17 @@ + + + BFT_SMaRt_TLS + + + + + + org.eclipse.jdt.core.javabuilder + + + + + + org.eclipse.jdt.core.javanature + + diff --git a/README.md b/README.md new file mode 100644 index 000000000..4a5354a65 --- /dev/null +++ b/README.md @@ -0,0 +1,101 @@ +# Byzantine Fault-Tolerant (BFT) State Machine Replication (SMaRt) v1.2 + +This is a Byzantine fault-tolerant state machine replication project named BFT-SMaRt, a Java open source library maintained by the LaSIGE research unit at the University of Lisbon. + +This package contains the source code (src/), jar file (bin/BFT-SMaRt.jar), dependencies (lib/), documentation (doc/), running scripts (runscripts/), and configuration files (config/) for version 1.2 of the project. +BFT-SMaRt requires the Java Runtime Environment version 1.8 or later. + +## Quick start + +To run any demonstration you first need to configure BFT-SMaRt to define the protocol behavior and the location of each replica. + +The servers must be specified in the configuration file (see `config/hosts.config`): + +``` +#server id, address and port (the ids from 0 to n-1 are the service replicas) +0 127.0.0.1 11000 11001 +1 127.0.0.1 11010 11011 +2 127.0.0.1 11020 11021 +3 127.0.0.1 11030 11031 +``` + +**Important tip #1:** Always provide IP addresses instead of hostnames. If a machine running a replica is not correctly configured, BFT-SMaRt may fail to bind to the appropriate IP address and use the loopback address instead (127.0.0.1). This phenomenom may prevent clients and/or replicas from successfully establishing a connection among them. + +**Important tip #2:** Clients requests should not be issued before all replicas have been properly initialized. Replicas are ready to process client requests when each one outputs `-- Ready to process operations` in the console. + +The system configurations also have to be specified (see`config/system.config`). Most of the parameters are self explanatory. + +**Important tip #3:** When using the library in real systems, always make sure to set `system.communication.defaultkeys` to `false` and `system.communication.useSignatures` to `1`. Also make sure that only the `config/keys` directory only has the private key for the repective replica/client. + +You can run the counter demonstration by executing the following commands, from within the main directory across four different consoles (4 replicas, to tolerate 1 fault): + +``` +./runscripts/smartrun.sh bftsmart.demo.counter.CounterServer 0 +./runscripts/smartrun.sh bftsmart.demo.counter.CounterServer 1 +./runscripts/smartrun.sh bftsmart.demo.counter.CounterServer 2 +./runscripts/smartrun.sh bftsmart.demo.counter.CounterServer 3 +``` + +**Important tip #4:** If you are getting timeout messages, it is possible that the application you are running takes too long to process the requests or the network delay is too high and PROPOSE messages from the leader does not arrive in time, so replicas may start the leader change protocol. To prevent that, try to increase the `system.totalordermulticast.timeout` parameter in 'config/system.config'. + +**Important tip #5:** Never forget to delete the `config/currentView` file after you modify `config/hosts.config` or `config/system.config`. If `config/currentView` exists, BFT-SMaRt always fetches the group configuration from this file first. Otherwise, BFT-SMaRt fetches information from the other files and creates `config/currentView` from scratch. Note that `config/currentView` only stores information related to the group of replicas. You do not need to delete this file if, for instance, you want to enable the debugger or change the value of the request timeout. + +Once all replicas are ready, the client can be launched as follows: + +``` +./runscripts/smartrun.sh bftsmart.demo.counter.CounterClient 1001 [] +``` + +If `` equals 0 the request will be read-only. Default `` equals 1000. + +**Important tip #6:** always make sure that each client uses a unique ID. Otherwise, clients may not be able to complete their operations. + +## State transfer protocol(s) + +BFT-SMaRt offers two state transfer protocols. The first is a basic protocol that can be used by extending the classes `bftsmart.tom.server.defaultservices.DefaultRecoverable` and `bftsmart.tom.server.defaultservices.DefaultSingleRecoverable`. Thee classes logs requests into memory and periodically takes snapshots of the application state. + +The second, more advanced protocol can be used by extending the class +`bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator`. This protocol stores its logs to disk. To mitigate the latency of writing to disk, such tasks is done in batches and in parallel with the requests' execution. Additionally, the snapshots are taken at different points of the execution in different replicas. + +**Important tip #7:** We recommend developers to use `bftsmart.tom.server.defaultservices.DefaultRecoverable`, since it is the most stable of the three classes. + +**Important tip #8:** regardless of the chosen protocol, developers must avoid using Java API objects like `HashSet` or `HashMap`, and use `TreeSet` or `TreeMap` instead. This is because serialization of Hash* objects is not deterministic, i.e, it generates different byte arrays for equal objects. This will lead to problems after more than `f` replicas used the state transfer protocol to recover from failures. + +## Group reconfiguration + +The library also implements a reconfiguration protocol that can be used to add/remove replicas from the initial group. You can add/remove replicas on-the-fly by executing the following commands: + +``` +./runscripts/smartrun.sh bftsmart.reconfiguration.util.DefaultVMServices (to add a replica to the group) +./runscripts/smartrun.sh bftsmart.reconfiguration.util.DefaultVMServices (to remove a replica from the group) +``` + +**Important tip #9:** everytime you use the reconfiguration protocol, you must make sure that all replicas and the host where you invoke the above commands have the latest `config/currentView` file. The current implementation of BFT-SMaRt does not provide any mechanism to distribute this file, so you will need to distribute it on your own (e.g., using the `scp` command). You also need to make sure that any client that starts executing can read from the latest `config/currentView` file. + +## BFT-SMaRt under crash faults + +You can run BFT-SMaRt in crash-faults only mode by setting the `system.bft` parameter in the configuration file to `false`. This mode requires less replicas to execute, but will not withstand full Byzantine behavior from compromised replicas. + +## Generating public/private key pairs + +If you need to generate public/private keys for more replicas or clients, you can use the following command: + +``` +./runscripts/smartrun.sh bftsmart.tom.util.RSAKeyPairGenerator +``` + +Keys are stored in the `config/keys` folder. The command above creates key pairs both for clients and replicas. Alternatively, you can set the `system.communication.defaultkeys` to `true` in the `config/system.config` file to forces all processes to use the same public/private keys pair and secret key. This is useful when deploying experiments and benchmarks, because it enables the programmer to avoid generating keys for all principals involved in the system. However, this must not be used in a real deployments. + +## Compiling + +Make sure that you have Ant installed and simply type `ant` in the main directory. The jar file is stored in the `bin/` directory. + +## Additional information and publications + +If you are interested in learning more about BFT-SMaRt, you can read: + +- The paper about its state machine protocol published in [EDCC'12](http://www.di.fc.ul.pt/~bessani/publications/edcc12-modsmart.pdf): +- The paper about its advanced state transfer protocol published in [Usenix'13](http://www.di.fc.ul.pt/~bessani/publications/usenix13-dsmr.pdf): +- The tool description published in [DSN'14](http://www.di.fc.ul.pt/~bessani/publications/dsn14-bftsmart.pdf): + +***Feel free to contact us if you have any questions!*** diff --git a/README.txt b/README.txt deleted file mode 100644 index 6aac5e5af..000000000 --- a/README.txt +++ /dev/null @@ -1,99 +0,0 @@ -BFT-SMaRt v1.1-beta ----------- - -This package contains the BFT-SMaRt source code (src/), binary file (bin/), libraries needed (lib/), documentation (doc/), running scripts (runscripts/) and configuration files (config/). -BFT-SMaRt requires the Java Runtime Environment version 1.7 or later. - ----------------- Important warning ------------------------ - -This beta version of BFT-SMaRt offers the most stable execution via the class bftsmart.tom.server.defaultservices.DefaultRecoverable under Byzantine faults. Applications can also be implemented using 'bftsmart.tom.server.defaultservices.DefaultSingleRecoverable' and 'bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator', but they are not as stable as 'DefaultRecoverable'. In future versions these classes will be properly tested and fixed. - -------------- How to run BFT-SMaRt ----------------------- - -To run any demonstration you first need to configure BFT-SMaRt to define the protocol behavior and the location of each replica. - -1.) The servers must be specified in the configuration file (see 'config/hosts.config'). An example: - -#server id, address and port (the ids from 0 to n-1 are the service replicas) -0 127.0.0.1 11000 -1 127.0.0.1 11010 -2 127.0.0.1 11020 -3 127.0.0.1 11030 - -Important tip #1: Always provide IP addresses instead of hostnames. If a machine running a replica is not correctly configured, BFT-SMaRt may fail to obtain the proper IP address and use the loopback address instead (127.0.0.1). This phenomenom may prevent clients and/or replicas from successfully establishing a connection among them. - -Important tip #2: If some (or all) replicas are deployed/executed within the same machine (127.0.0.1), 'config/hosts.config' cannot have sequencial port numbers (e.g., 10000, 10001, 10002, 10003). This is because each replica binds two ports: one to receive messages from clients (that are configured in 'config/hosts.config', as shown above) and other to receive message from the other replicas (chosen by getting the next port number). More generally, if replica R is assigned port number P, it will try to bind ports P (to received client requests) and P+1 (to communicate with other replicas). If this guideline is not enforced, replicas may not be able to bind all ports that are needed. - -Important tip #3: Clients requests should not be issued before all replicas have been properly initialized. Replicas are ready to process client requests when each one outputs '(DeliveryThread.run) canDeliver released.' in the console. - -2.) The system configurations also have to be specified (see 'config/system.config'). Most of the parameters are self explanatory. - -You can run the counter demonstration by executing the following commands, from within the main folder: - -#Start the servers (4 replicas, to tolerate 1 fault) -./runscripts/smartrun.sh bftsmart.demo.counter.CounterServer 0 -./runscripts/smartrun.sh bftsmart.demo.counter.CounterServer 1 -./runscripts/smartrun.sh bftsmart.demo.counter.CounterServer 2 -./runscripts/smartrun.sh bftsmart.demo.counter.CounterServer 3 - -Important tip #4: If you are getting timeout messages, it is possible that the application you are running takes too long to process the requests or the network delay is too high and PROPOSE messages from the leader don't arrive in time, so replicas may start the leader change protocol. To prevent that, try to increase the 'system.totalordermulticast.timeout' parameter in 'config/system.config'. - -Important tip #5: Never forget to delete the 'config/currentView' file after you modify 'config/hosts.config' or 'config/system.config'. If 'config/currentView' exists, BFT-SMaRt always fetches the group configuration from this file first. Otherwise, BFT-SMaRt fetches information from the other files and creates 'config/currentView' from scratch. Note that 'config/currentView' only stores information related to the group of replicas. You do not need to delete this file if, for instance, you want to enable the debugger or change the value of the request timeout. - -#Start a client - -./runscripts/smartrun.sh bftsmart.demo.counter.CounterClient 1001 [] - -If equals 0 the request will be read-only. Default equals 1000. - -Important tip #6: always make sure that each client uses a unique ID. Otherwise, clients may not be able to complete their operations. - -You can use the './runscripts/runsmart.bat'" script in Windows, and the './runscripts/runsmart.sh' script in Linux. -When running the script in Linux it is necessary to set the permissions to execute the script with the command 'chmod +x ./runscripts/runsmart.sh'. - -These scripts can easily be adapted to execute other demos, such as: - -- Random. You can run it by using the 'RandomServer' and 'RandomClient' classes located in the package 'bftsmart.demo.random'. -- BFTMap. A Table of hash maps where tables can be created and key value pair added to it. - The server is 'bftmap.demo.bftmap.BFTMapServer' and the clients are 'BFTMapClient' for incremental inserts or 'BFTMapInteractiveClient' for a command line client. Parameters to run the BFTMap demo are displayed when attempts to start the servers and clients are made without parameters. -- YCSB. You can run a Yahoo! Cloud Serving Benchmark with BFT-SMaRt by executing the './runscripts/startReplicaYCSB.sh' and './runscripts/ycsbClient.sh' scripts. - ----------- State transfer protocol(s) -------------- - -BFT-SMaRt offers two state transfer protocols. The first is a basic protocol that can be used by extending the class 'bftsmart.tom.server.defaultservices.DefaultRecoverable' that logs requests into memory and periodically takes snapshots of the application state. - -The second, more advanced protocol can be used by extending the class 'bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator'. This protocol stores its logs to disk. To mitigate the latency of writing to disk, such tasks is done in batches and in parallel with the requests' execution. Additionally, the snapshots are taken at different points of the execution in different replicas. - -Important tip #7: regardless of the chosen protocol, developers must avoid using Java API objects like 'HashSet' or 'HashMap', and use 'TreeSet' or 'TreeMap' instead. This is because serialization of Hash* objects is not deterministic, i.e, it generates different results for equal objects. This will lead to problems after more than 'f' replicas used the state transfer protocol to recover from failures. - ------------ Group reconfiguration ------------------ - -The library also implements a reconfiguration protocol that can be used to add/remove replicas from the initial group. You can add/remove replicas on-the-fly by executing the following commands: - -./runscripts/smartrun.sh bftsmart.reconfiguration.util.DefaultVMServices (to add a replica to the group) -./runscripts/smartrun.sh bftsmart.reconfiguration.util.DefaultVMServices (to remove a replica from the group) - -Important tip #8: everytime you use the reconfiguration protocol, you must make sure that all replicas and the host where you invoke the above commands have the latest 'config/currentView' file. The current implementation of BFT-SMaRt does not provide any mechanism to distribute this file, so you will need to distribute it on your own (e.g., using the 'scp' command). You also need to make sure that any client that starts executing can read from the latest config/currentView file. - ----------- BFT-SMaRt under crash faults ------------ - -You can run BFT-SMaRt in crash-faults only mode by setting the 'system.bft' parameter in the configuration file to 'false'. This mode requires less replicas to execute, but will not withstand full Byzantine behavior from compromised replicas. - ------- Generating public/private key pairs --------- - -If you need to generate public/private keys for more replicas or clients, you can use the following command: - -./runscripts/smartrun.sh bftsmart.tom.util.RSAKeyPairGenerator - -Keys are stored in the 'config/keys' folder. The command above creates key pairs both for clients and replicas. Alternatively, you can set the 'system.communication.defaultkeys' to 'true' in the 'config/system.config' file to forces all processes to use the same public/private keys pair and secret key. This is useful when deploying experiments and benchmarks, because it enables the programmer to avoid generating keys for all principals involved in the system. However, this must not be used in a real deployment. - ------ Additional information and publications ------ - -Finally, if you are interested in learning more about BFT-SMaRt, you can read: - -- The technical report at the handler: http://hdl.handle.net/10451/14170 -- The paper about its state machine protocol published in EDCC'12: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6214759 -- The paper about its advanced state transfer protocol published in Usenix'13: https://www.usenix.org/conference/atc13/technical-sessions/presentation/bessani -- The tool description published in DSN'14: http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6903593&tag=1 - -Feel free to contact us if you have any questions! diff --git a/bin/BFT-SMaRt.jar b/bin/BFT-SMaRt.jar deleted file mode 100644 index f900326fe..000000000 Binary files a/bin/BFT-SMaRt.jar and /dev/null differ diff --git a/changes.txt b/changes.txt new file mode 100644 index 000000000..c87174520 --- /dev/null +++ b/changes.txt @@ -0,0 +1,21 @@ +First changes (commit), for porting to TLS. + ++ Altered system.config, added TLS configurations. ++ Added directory with specific keys. ++ Updated Netty from 4.1.30 to 4.1.34. + +Second changes ++ Changed communication system. ++ Changed ServerConnection ++ Changed Configuration and TOMConfiguration. ++ Added BouncyCastle provider at TOMUtil.init(). ++ Tested changes, seems working. +TODO: Test failure scenarios. + + +Third changes ++ Remove hmac and mac from code and system.config ++ Decreased creation time among threads (ThroughputLatencyClient) + +TODO: + diff --git a/config/hosts.config b/config/hosts.config index da163861c..01f78b957 100644 --- a/config/hosts.config +++ b/config/hosts.config @@ -18,19 +18,21 @@ # The ports defined here are the ports used by clients to communicate # with the replicas. Additional connections are opened by replicas to # communicate with each other. This additional connection is opened in the -# next port defined here. For an example, consider the line "0 127.0.0.1 11000". +# next port defined here. For an example, consider the line "0 127.0.0.1 11000 11001". # That means that clients will open a communication channel to replica 0 in # IP 127.0.0.1 and port 11000. On startup, replicas with id different than 0 # will open a communication channel to replica 0 in port 11001. # The same holds for replicas 1, 2, 3 ... N. #server id, address and port (the ids from 0 to n-1 are the service replicas) -0 127.0.0.1 11000 -1 127.0.0.1 11010 -2 127.0.0.1 11020 -3 127.0.0.1 11030 -4 127.0.0.1 11040 -5 127.0.0.1 11050 -6 127.0.0.1 11060 -7 127.0.0.1 11070 +0 127.0.0.1 11000 11001 +1 127.0.0.1 11010 11011 +2 127.0.0.1 11020 11021 +3 127.0.0.1 11030 11031 + +#0 192.168.2.29 11000 11001 +#1 192.168.2.30 11000 11001 +#2 192.168.2.31 11000 11001 +#3 192.168.2.32 11000 11001 + 7001 127.0.0.1 11100 diff --git a/config/keys/privatekey1002 b/config/keys/privatekey1002 deleted file mode 100644 index 2dc2d9874..000000000 --- a/config/keys/privatekey1002 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAIOsuw2PdzI2NvWA81iltnmj+6Xn -Un1sMQ8kNro6Z9h0vrEjwccjtqsAw5KupzLFjgZBZlld3xwCiKdvXH0C5Lml9hCB3vSolnb3oWyU -NyS59TvwY0yB/IM+ZYICO+b2LLoo+72DR0HUPbWQOXXAg4QFNCIBVi3G+hI1aO2I3e63AgMBAAEC -gYBv0O3jXRJGhH11S7TacZacg2F+iAqn0YlzpzgGOvTWfoU03f1/Q1eqrXJnDIOVfOfXbhAhEbMW -ERzEuq+axSs/LcjYSlj1PyWbKm0OoiVQS9YklqVl2oQyN7mrUspRBnvq35M33+cOec/0BnbXjl8L -H2N2q5xC/7b76U83ktKJgQJBAOyNtcwcsY0J+NZNJQwdpW4hmOriy672AZjAZ7spSrXngF3rTrqP -N/dIczudLYVc6Ws04aJdHnadlR4TMfqcZUECQQCOf9YuM8pSdqVsM4cS/E8DvmbM1w7OY3HWYuZz -4wsWWylDbiBu0ETsGC2M+ojfd4Ap6/DPayT/g1GBZRn/J/33AkEA6QjJmaSCvQCf5O3mm0LxX8c1 -T9/Q7DFpbUPObLDG6uB+swdGbb79UVxMOOQkngbj5DsaNMsrYYroBdK/H7XGgQJAFuyL2SztvGxn -2ktDzJMYZwLHaOaj9CHdW17XsgbxbqxFrBpYT1tj6xNMqKRZRpHj5VqWh9e1wQEwKH/KOdkGKwJB -AOEO/Xs8BWnzF/vsXGN7E/xiFOhio4mP862NqQSL8cxRmswzMeLJ3cdv1Z1lxXGFIY+aOczB6Sm4 -jnszzWPp0yA= \ No newline at end of file diff --git a/config/keys/privatekey1003 b/config/keys/privatekey1003 deleted file mode 100644 index 2dc2d9874..000000000 --- a/config/keys/privatekey1003 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAIOsuw2PdzI2NvWA81iltnmj+6Xn -Un1sMQ8kNro6Z9h0vrEjwccjtqsAw5KupzLFjgZBZlld3xwCiKdvXH0C5Lml9hCB3vSolnb3oWyU -NyS59TvwY0yB/IM+ZYICO+b2LLoo+72DR0HUPbWQOXXAg4QFNCIBVi3G+hI1aO2I3e63AgMBAAEC -gYBv0O3jXRJGhH11S7TacZacg2F+iAqn0YlzpzgGOvTWfoU03f1/Q1eqrXJnDIOVfOfXbhAhEbMW -ERzEuq+axSs/LcjYSlj1PyWbKm0OoiVQS9YklqVl2oQyN7mrUspRBnvq35M33+cOec/0BnbXjl8L -H2N2q5xC/7b76U83ktKJgQJBAOyNtcwcsY0J+NZNJQwdpW4hmOriy672AZjAZ7spSrXngF3rTrqP -N/dIczudLYVc6Ws04aJdHnadlR4TMfqcZUECQQCOf9YuM8pSdqVsM4cS/E8DvmbM1w7OY3HWYuZz -4wsWWylDbiBu0ETsGC2M+ojfd4Ap6/DPayT/g1GBZRn/J/33AkEA6QjJmaSCvQCf5O3mm0LxX8c1 -T9/Q7DFpbUPObLDG6uB+swdGbb79UVxMOOQkngbj5DsaNMsrYYroBdK/H7XGgQJAFuyL2SztvGxn -2ktDzJMYZwLHaOaj9CHdW17XsgbxbqxFrBpYT1tj6xNMqKRZRpHj5VqWh9e1wQEwKH/KOdkGKwJB -AOEO/Xs8BWnzF/vsXGN7E/xiFOhio4mP862NqQSL8cxRmswzMeLJ3cdv1Z1lxXGFIY+aOczB6Sm4 -jnszzWPp0yA= \ No newline at end of file diff --git a/config/keys/privatekey1004 b/config/keys/privatekey1004 deleted file mode 100644 index 2dc2d9874..000000000 --- a/config/keys/privatekey1004 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAIOsuw2PdzI2NvWA81iltnmj+6Xn -Un1sMQ8kNro6Z9h0vrEjwccjtqsAw5KupzLFjgZBZlld3xwCiKdvXH0C5Lml9hCB3vSolnb3oWyU -NyS59TvwY0yB/IM+ZYICO+b2LLoo+72DR0HUPbWQOXXAg4QFNCIBVi3G+hI1aO2I3e63AgMBAAEC -gYBv0O3jXRJGhH11S7TacZacg2F+iAqn0YlzpzgGOvTWfoU03f1/Q1eqrXJnDIOVfOfXbhAhEbMW -ERzEuq+axSs/LcjYSlj1PyWbKm0OoiVQS9YklqVl2oQyN7mrUspRBnvq35M33+cOec/0BnbXjl8L -H2N2q5xC/7b76U83ktKJgQJBAOyNtcwcsY0J+NZNJQwdpW4hmOriy672AZjAZ7spSrXngF3rTrqP -N/dIczudLYVc6Ws04aJdHnadlR4TMfqcZUECQQCOf9YuM8pSdqVsM4cS/E8DvmbM1w7OY3HWYuZz -4wsWWylDbiBu0ETsGC2M+ojfd4Ap6/DPayT/g1GBZRn/J/33AkEA6QjJmaSCvQCf5O3mm0LxX8c1 -T9/Q7DFpbUPObLDG6uB+swdGbb79UVxMOOQkngbj5DsaNMsrYYroBdK/H7XGgQJAFuyL2SztvGxn -2ktDzJMYZwLHaOaj9CHdW17XsgbxbqxFrBpYT1tj6xNMqKRZRpHj5VqWh9e1wQEwKH/KOdkGKwJB -AOEO/Xs8BWnzF/vsXGN7E/xiFOhio4mP862NqQSL8cxRmswzMeLJ3cdv1Z1lxXGFIY+aOczB6Sm4 -jnszzWPp0yA= \ No newline at end of file diff --git a/config/keys/privatekey1005 b/config/keys/privatekey1005 deleted file mode 100644 index d5e66554e..000000000 --- a/config/keys/privatekey1005 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKSf6ycmjnW3EXqYG+kPKXmtBZ0s -ZPzAqyMYdXwB6OZ9VDO5KmZH6tpnAGrCVu0kSXDLJxEctXWpaMKMEWZ+nc3saiayaPHUKueQNrEn -zIBwanLjLpLQtBYONkugYFZroSMnISB+Ruqv0gJlOxaS62PZcsNdbxmkAGM1qPU5UXn9AgMBAAEC -gYAs5fiqeteLv1wjBW1HSP1U1B2d2kfIr3t9XORd599vTWszmCK6No8U9dalR0+R6b2l56JGBoBv -EL8joDP2sj2zRR/O1yybvJhWGts/7efHb5T8eraDus+btRlOlwJTfYaLFVmf+eNauaqydgNiv858 -TYfq7hXmYlDRtN11WuvSuQJBAOf/SprCoutduU2Z+YDMmN+t8v0k8DXxSdyyonKhAr85g8p6W6/c -+RylLouRROwqQvIfxnRvbZf68pvKib/zB0MCQQC1qDBouQdsR+K8KXer7kTJBIMccuYotMhm84Yc -3MRHZvR3tfXTJlyFQw3f8FT4sKChmWQA2Rv1sIku1aa8LkW/AkEAhrV/9ljqDMzBGkQ70gP7CHaS -z1axfED86H95QCIp3CtjUSC3OGiQ5YcsfbH+WTs44Cp/K4DAHC2Ezd1PGvvczQJAfesbwzuevZwG -us8zuOXsoDqUd3/dspG96D8d05vkImpnQyPKRpbswyL71yLMg6ZLcjjVbRERpHi+XsySSxb4GQJA -NUK1DEc615gggvYWgSYGjj7XxhrDhhxifukB3eCEWavaDJD8iXQTcmL+64/lTH24Yq10cmDm+d24 -T0aFR1RoPQ== \ No newline at end of file diff --git a/config/keys/privatekey1006 b/config/keys/privatekey1006 deleted file mode 100644 index d5e66554e..000000000 --- a/config/keys/privatekey1006 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKSf6ycmjnW3EXqYG+kPKXmtBZ0s -ZPzAqyMYdXwB6OZ9VDO5KmZH6tpnAGrCVu0kSXDLJxEctXWpaMKMEWZ+nc3saiayaPHUKueQNrEn -zIBwanLjLpLQtBYONkugYFZroSMnISB+Ruqv0gJlOxaS62PZcsNdbxmkAGM1qPU5UXn9AgMBAAEC -gYAs5fiqeteLv1wjBW1HSP1U1B2d2kfIr3t9XORd599vTWszmCK6No8U9dalR0+R6b2l56JGBoBv -EL8joDP2sj2zRR/O1yybvJhWGts/7efHb5T8eraDus+btRlOlwJTfYaLFVmf+eNauaqydgNiv858 -TYfq7hXmYlDRtN11WuvSuQJBAOf/SprCoutduU2Z+YDMmN+t8v0k8DXxSdyyonKhAr85g8p6W6/c -+RylLouRROwqQvIfxnRvbZf68pvKib/zB0MCQQC1qDBouQdsR+K8KXer7kTJBIMccuYotMhm84Yc -3MRHZvR3tfXTJlyFQw3f8FT4sKChmWQA2Rv1sIku1aa8LkW/AkEAhrV/9ljqDMzBGkQ70gP7CHaS -z1axfED86H95QCIp3CtjUSC3OGiQ5YcsfbH+WTs44Cp/K4DAHC2Ezd1PGvvczQJAfesbwzuevZwG -us8zuOXsoDqUd3/dspG96D8d05vkImpnQyPKRpbswyL71yLMg6ZLcjjVbRERpHi+XsySSxb4GQJA -NUK1DEc615gggvYWgSYGjj7XxhrDhhxifukB3eCEWavaDJD8iXQTcmL+64/lTH24Yq10cmDm+d24 -T0aFR1RoPQ== \ No newline at end of file diff --git a/config/keys/privatekey1007 b/config/keys/privatekey1007 deleted file mode 100644 index 4b498b83f..000000000 --- a/config/keys/privatekey1007 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAJTlQg0vlwe6dPOffFxPjTk4fubI -wu/Gw6cxckSU9cXP26qBcof57iJyBI+0tQVGVOSarXFFY1AstYNq/DmhBJLRgtVoTFTheuZqgsdT -QT5uBEdWij/fj3R4jlCqZ3T4N2jwJnGK1JVoM6b8WGC+Fu8Wyd6qMnNirCSPy/g0NdytAgMBAAEC -gYA0okJSJPOPsNLs2AD/JMP4aUc6z9dRKYM6VBlfswSac5rkUDQraydKF+IS5r9i6PTThvYu52dv -DQ7xMN2OhaP2tC/U4HM6Bz5jw4inlpUoHolmVaTp3isqo6XMj2vMLkTeLOE9H/l/fp7QaZ383p6+ -otFavgp4cXKuTKv2/buqhQJBANC/NS3i73w3ypswycGfL6pHAcZTUx5nIuJ4ue5jVL0ZX4MTMZVw -cWv9u+ZYp2vgZhGtCDFLVBEZhKGdNenc0AsCQQC2mbCOqEQ05AsuHYXWW2fum78xQecvLxg3Nqjm -BmgZx0EzAHbl1i87pXsTipYemg/CiQkAHyy2Xw5lNmX6+mEnAkBNoD0zYieypsCmM19jrEG6bYtg -aA4JyUq0szZZSwDxDSVyhcmzi8vx37pnL2KbACiUvCbGxf9uGHI081RgMif/AkEAk8t/yWfOlRTD -K9c/fcnMLKgpL0BNU7aFniMCv/QNjFfnoNVOZydeYaPGI4H3kdtWyDlMbKnO8emsaCWIk4TW3wJA -BZAI8/aRcyqFAKyui29f3nvyMQ5fgOaJRN4nsiPT+inFiIRmvd6KL/3eBrk0y4d73CKOVET4Bdm1 -xGh1TAE27Q== \ No newline at end of file diff --git a/config/keys/privatekey1008 b/config/keys/privatekey1008 deleted file mode 100644 index 4b498b83f..000000000 --- a/config/keys/privatekey1008 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAJTlQg0vlwe6dPOffFxPjTk4fubI -wu/Gw6cxckSU9cXP26qBcof57iJyBI+0tQVGVOSarXFFY1AstYNq/DmhBJLRgtVoTFTheuZqgsdT -QT5uBEdWij/fj3R4jlCqZ3T4N2jwJnGK1JVoM6b8WGC+Fu8Wyd6qMnNirCSPy/g0NdytAgMBAAEC -gYA0okJSJPOPsNLs2AD/JMP4aUc6z9dRKYM6VBlfswSac5rkUDQraydKF+IS5r9i6PTThvYu52dv -DQ7xMN2OhaP2tC/U4HM6Bz5jw4inlpUoHolmVaTp3isqo6XMj2vMLkTeLOE9H/l/fp7QaZ383p6+ -otFavgp4cXKuTKv2/buqhQJBANC/NS3i73w3ypswycGfL6pHAcZTUx5nIuJ4ue5jVL0ZX4MTMZVw -cWv9u+ZYp2vgZhGtCDFLVBEZhKGdNenc0AsCQQC2mbCOqEQ05AsuHYXWW2fum78xQecvLxg3Nqjm -BmgZx0EzAHbl1i87pXsTipYemg/CiQkAHyy2Xw5lNmX6+mEnAkBNoD0zYieypsCmM19jrEG6bYtg -aA4JyUq0szZZSwDxDSVyhcmzi8vx37pnL2KbACiUvCbGxf9uGHI081RgMif/AkEAk8t/yWfOlRTD -K9c/fcnMLKgpL0BNU7aFniMCv/QNjFfnoNVOZydeYaPGI4H3kdtWyDlMbKnO8emsaCWIk4TW3wJA -BZAI8/aRcyqFAKyui29f3nvyMQ5fgOaJRN4nsiPT+inFiIRmvd6KL/3eBrk0y4d73CKOVET4Bdm1 -xGh1TAE27Q== \ No newline at end of file diff --git a/config/keys/privatekey1009 b/config/keys/privatekey1009 deleted file mode 100644 index 6971acfac..000000000 --- a/config/keys/privatekey1009 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBALcyCISI8Yxfdb86aPjyLCIU4qlK -loZ/uWp9W9BiXCSUGkPrgLtdF6j24siUL9V5d6TGmKuUA11guKsUTwpQJUcEGP7LJw80B5sViiXG -DHlnBkGkolsyupKAsajlo/rNY2dzWwMiskwgCf13DWBiExV13B1gRtQt/EEWYV63PgfjAgMBAAEC -gYAmMHnyFxIzrxq+uJB1hRaKLExr+i6S70FZ37QqJrcKsQ0gp5F4fZmXEZCJO8sU93NnmDbvXbWt -26HjmCI6Bice4jPjdB5ZkfZ5dpwqJuuOp0mtivfVjk8hrW3LjagnLOHYzTcIrgveU6L5M3oG6e91 -ilLDF1nJ81cfnM5XSVA7oQJBAOtJ1VcGCZgeMCNfCdC/KViWPm7AaSqMr67BuZpePMWYoi5Sm3pg -PVUV89sDd1GcGlWb31gmFha0udNlC+DbGDsCQQDHUkwxpl48AmM+Nu/6cRKxJy5CL7x5y1Lmok2Y -Ldq3wt54pAITpGLS37WxHWQGQ6X77kr1fFj1fkNkwKjjqXx5AkEAs/Ix0W5yo/5iforO7PFmY7P0 -Z6uMQth5/EN4+ZCdFdoYHylac4trHMm/SH8sICQtF9AwTjBCsHbjr8MWBJBLxQJAUmu5jh5ZgT8L -gsJ84Xa4YADvo/bRXRuyoCQlKfRsKOivzfsj0LN9Ge3f1+lrXxvCGzciT/inYHxhqdvw+pMvIQJA -FVo6Q4VYj1jDe9zKmxL91WUwt85/dvSNcCayNaT5moC4GRFIkWwojsSbN0SCyMhti6u38OCZy1dd -NOhf3eXFXQ== \ No newline at end of file diff --git a/config/keys/privatekey1010 b/config/keys/privatekey1010 deleted file mode 100644 index 6971acfac..000000000 --- a/config/keys/privatekey1010 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBALcyCISI8Yxfdb86aPjyLCIU4qlK -loZ/uWp9W9BiXCSUGkPrgLtdF6j24siUL9V5d6TGmKuUA11guKsUTwpQJUcEGP7LJw80B5sViiXG -DHlnBkGkolsyupKAsajlo/rNY2dzWwMiskwgCf13DWBiExV13B1gRtQt/EEWYV63PgfjAgMBAAEC -gYAmMHnyFxIzrxq+uJB1hRaKLExr+i6S70FZ37QqJrcKsQ0gp5F4fZmXEZCJO8sU93NnmDbvXbWt -26HjmCI6Bice4jPjdB5ZkfZ5dpwqJuuOp0mtivfVjk8hrW3LjagnLOHYzTcIrgveU6L5M3oG6e91 -ilLDF1nJ81cfnM5XSVA7oQJBAOtJ1VcGCZgeMCNfCdC/KViWPm7AaSqMr67BuZpePMWYoi5Sm3pg -PVUV89sDd1GcGlWb31gmFha0udNlC+DbGDsCQQDHUkwxpl48AmM+Nu/6cRKxJy5CL7x5y1Lmok2Y -Ldq3wt54pAITpGLS37WxHWQGQ6X77kr1fFj1fkNkwKjjqXx5AkEAs/Ix0W5yo/5iforO7PFmY7P0 -Z6uMQth5/EN4+ZCdFdoYHylac4trHMm/SH8sICQtF9AwTjBCsHbjr8MWBJBLxQJAUmu5jh5ZgT8L -gsJ84Xa4YADvo/bRXRuyoCQlKfRsKOivzfsj0LN9Ge3f1+lrXxvCGzciT/inYHxhqdvw+pMvIQJA -FVo6Q4VYj1jDe9zKmxL91WUwt85/dvSNcCayNaT5moC4GRFIkWwojsSbN0SCyMhti6u38OCZy1dd -NOhf3eXFXQ== \ No newline at end of file diff --git a/config/keys/privatekey1011 b/config/keys/privatekey1011 deleted file mode 100644 index 2dc2d9874..000000000 --- a/config/keys/privatekey1011 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAIOsuw2PdzI2NvWA81iltnmj+6Xn -Un1sMQ8kNro6Z9h0vrEjwccjtqsAw5KupzLFjgZBZlld3xwCiKdvXH0C5Lml9hCB3vSolnb3oWyU -NyS59TvwY0yB/IM+ZYICO+b2LLoo+72DR0HUPbWQOXXAg4QFNCIBVi3G+hI1aO2I3e63AgMBAAEC -gYBv0O3jXRJGhH11S7TacZacg2F+iAqn0YlzpzgGOvTWfoU03f1/Q1eqrXJnDIOVfOfXbhAhEbMW -ERzEuq+axSs/LcjYSlj1PyWbKm0OoiVQS9YklqVl2oQyN7mrUspRBnvq35M33+cOec/0BnbXjl8L -H2N2q5xC/7b76U83ktKJgQJBAOyNtcwcsY0J+NZNJQwdpW4hmOriy672AZjAZ7spSrXngF3rTrqP -N/dIczudLYVc6Ws04aJdHnadlR4TMfqcZUECQQCOf9YuM8pSdqVsM4cS/E8DvmbM1w7OY3HWYuZz -4wsWWylDbiBu0ETsGC2M+ojfd4Ap6/DPayT/g1GBZRn/J/33AkEA6QjJmaSCvQCf5O3mm0LxX8c1 -T9/Q7DFpbUPObLDG6uB+swdGbb79UVxMOOQkngbj5DsaNMsrYYroBdK/H7XGgQJAFuyL2SztvGxn -2ktDzJMYZwLHaOaj9CHdW17XsgbxbqxFrBpYT1tj6xNMqKRZRpHj5VqWh9e1wQEwKH/KOdkGKwJB -AOEO/Xs8BWnzF/vsXGN7E/xiFOhio4mP862NqQSL8cxRmswzMeLJ3cdv1Z1lxXGFIY+aOczB6Sm4 -jnszzWPp0yA= \ No newline at end of file diff --git a/config/keys/privatekey1012 b/config/keys/privatekey1012 deleted file mode 100644 index 2dc2d9874..000000000 --- a/config/keys/privatekey1012 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAIOsuw2PdzI2NvWA81iltnmj+6Xn -Un1sMQ8kNro6Z9h0vrEjwccjtqsAw5KupzLFjgZBZlld3xwCiKdvXH0C5Lml9hCB3vSolnb3oWyU -NyS59TvwY0yB/IM+ZYICO+b2LLoo+72DR0HUPbWQOXXAg4QFNCIBVi3G+hI1aO2I3e63AgMBAAEC -gYBv0O3jXRJGhH11S7TacZacg2F+iAqn0YlzpzgGOvTWfoU03f1/Q1eqrXJnDIOVfOfXbhAhEbMW -ERzEuq+axSs/LcjYSlj1PyWbKm0OoiVQS9YklqVl2oQyN7mrUspRBnvq35M33+cOec/0BnbXjl8L -H2N2q5xC/7b76U83ktKJgQJBAOyNtcwcsY0J+NZNJQwdpW4hmOriy672AZjAZ7spSrXngF3rTrqP -N/dIczudLYVc6Ws04aJdHnadlR4TMfqcZUECQQCOf9YuM8pSdqVsM4cS/E8DvmbM1w7OY3HWYuZz -4wsWWylDbiBu0ETsGC2M+ojfd4Ap6/DPayT/g1GBZRn/J/33AkEA6QjJmaSCvQCf5O3mm0LxX8c1 -T9/Q7DFpbUPObLDG6uB+swdGbb79UVxMOOQkngbj5DsaNMsrYYroBdK/H7XGgQJAFuyL2SztvGxn -2ktDzJMYZwLHaOaj9CHdW17XsgbxbqxFrBpYT1tj6xNMqKRZRpHj5VqWh9e1wQEwKH/KOdkGKwJB -AOEO/Xs8BWnzF/vsXGN7E/xiFOhio4mP862NqQSL8cxRmswzMeLJ3cdv1Z1lxXGFIY+aOczB6Sm4 -jnszzWPp0yA= \ No newline at end of file diff --git a/config/keys/privatekey1013 b/config/keys/privatekey1013 deleted file mode 100644 index 2dc2d9874..000000000 --- a/config/keys/privatekey1013 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAIOsuw2PdzI2NvWA81iltnmj+6Xn -Un1sMQ8kNro6Z9h0vrEjwccjtqsAw5KupzLFjgZBZlld3xwCiKdvXH0C5Lml9hCB3vSolnb3oWyU -NyS59TvwY0yB/IM+ZYICO+b2LLoo+72DR0HUPbWQOXXAg4QFNCIBVi3G+hI1aO2I3e63AgMBAAEC -gYBv0O3jXRJGhH11S7TacZacg2F+iAqn0YlzpzgGOvTWfoU03f1/Q1eqrXJnDIOVfOfXbhAhEbMW -ERzEuq+axSs/LcjYSlj1PyWbKm0OoiVQS9YklqVl2oQyN7mrUspRBnvq35M33+cOec/0BnbXjl8L -H2N2q5xC/7b76U83ktKJgQJBAOyNtcwcsY0J+NZNJQwdpW4hmOriy672AZjAZ7spSrXngF3rTrqP -N/dIczudLYVc6Ws04aJdHnadlR4TMfqcZUECQQCOf9YuM8pSdqVsM4cS/E8DvmbM1w7OY3HWYuZz -4wsWWylDbiBu0ETsGC2M+ojfd4Ap6/DPayT/g1GBZRn/J/33AkEA6QjJmaSCvQCf5O3mm0LxX8c1 -T9/Q7DFpbUPObLDG6uB+swdGbb79UVxMOOQkngbj5DsaNMsrYYroBdK/H7XGgQJAFuyL2SztvGxn -2ktDzJMYZwLHaOaj9CHdW17XsgbxbqxFrBpYT1tj6xNMqKRZRpHj5VqWh9e1wQEwKH/KOdkGKwJB -AOEO/Xs8BWnzF/vsXGN7E/xiFOhio4mP862NqQSL8cxRmswzMeLJ3cdv1Z1lxXGFIY+aOczB6Sm4 -jnszzWPp0yA= \ No newline at end of file diff --git a/config/keys/privatekey1014 b/config/keys/privatekey1014 deleted file mode 100644 index 2dc2d9874..000000000 --- a/config/keys/privatekey1014 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAIOsuw2PdzI2NvWA81iltnmj+6Xn -Un1sMQ8kNro6Z9h0vrEjwccjtqsAw5KupzLFjgZBZlld3xwCiKdvXH0C5Lml9hCB3vSolnb3oWyU -NyS59TvwY0yB/IM+ZYICO+b2LLoo+72DR0HUPbWQOXXAg4QFNCIBVi3G+hI1aO2I3e63AgMBAAEC -gYBv0O3jXRJGhH11S7TacZacg2F+iAqn0YlzpzgGOvTWfoU03f1/Q1eqrXJnDIOVfOfXbhAhEbMW -ERzEuq+axSs/LcjYSlj1PyWbKm0OoiVQS9YklqVl2oQyN7mrUspRBnvq35M33+cOec/0BnbXjl8L -H2N2q5xC/7b76U83ktKJgQJBAOyNtcwcsY0J+NZNJQwdpW4hmOriy672AZjAZ7spSrXngF3rTrqP -N/dIczudLYVc6Ws04aJdHnadlR4TMfqcZUECQQCOf9YuM8pSdqVsM4cS/E8DvmbM1w7OY3HWYuZz -4wsWWylDbiBu0ETsGC2M+ojfd4Ap6/DPayT/g1GBZRn/J/33AkEA6QjJmaSCvQCf5O3mm0LxX8c1 -T9/Q7DFpbUPObLDG6uB+swdGbb79UVxMOOQkngbj5DsaNMsrYYroBdK/H7XGgQJAFuyL2SztvGxn -2ktDzJMYZwLHaOaj9CHdW17XsgbxbqxFrBpYT1tj6xNMqKRZRpHj5VqWh9e1wQEwKH/KOdkGKwJB -AOEO/Xs8BWnzF/vsXGN7E/xiFOhio4mP862NqQSL8cxRmswzMeLJ3cdv1Z1lxXGFIY+aOczB6Sm4 -jnszzWPp0yA= \ No newline at end of file diff --git a/config/keys/privatekey1015 b/config/keys/privatekey1015 deleted file mode 100644 index d5e66554e..000000000 --- a/config/keys/privatekey1015 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKSf6ycmjnW3EXqYG+kPKXmtBZ0s -ZPzAqyMYdXwB6OZ9VDO5KmZH6tpnAGrCVu0kSXDLJxEctXWpaMKMEWZ+nc3saiayaPHUKueQNrEn -zIBwanLjLpLQtBYONkugYFZroSMnISB+Ruqv0gJlOxaS62PZcsNdbxmkAGM1qPU5UXn9AgMBAAEC -gYAs5fiqeteLv1wjBW1HSP1U1B2d2kfIr3t9XORd599vTWszmCK6No8U9dalR0+R6b2l56JGBoBv -EL8joDP2sj2zRR/O1yybvJhWGts/7efHb5T8eraDus+btRlOlwJTfYaLFVmf+eNauaqydgNiv858 -TYfq7hXmYlDRtN11WuvSuQJBAOf/SprCoutduU2Z+YDMmN+t8v0k8DXxSdyyonKhAr85g8p6W6/c -+RylLouRROwqQvIfxnRvbZf68pvKib/zB0MCQQC1qDBouQdsR+K8KXer7kTJBIMccuYotMhm84Yc -3MRHZvR3tfXTJlyFQw3f8FT4sKChmWQA2Rv1sIku1aa8LkW/AkEAhrV/9ljqDMzBGkQ70gP7CHaS -z1axfED86H95QCIp3CtjUSC3OGiQ5YcsfbH+WTs44Cp/K4DAHC2Ezd1PGvvczQJAfesbwzuevZwG -us8zuOXsoDqUd3/dspG96D8d05vkImpnQyPKRpbswyL71yLMg6ZLcjjVbRERpHi+XsySSxb4GQJA -NUK1DEc615gggvYWgSYGjj7XxhrDhhxifukB3eCEWavaDJD8iXQTcmL+64/lTH24Yq10cmDm+d24 -T0aFR1RoPQ== \ No newline at end of file diff --git a/config/keys/privatekey1016 b/config/keys/privatekey1016 deleted file mode 100644 index d5e66554e..000000000 --- a/config/keys/privatekey1016 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKSf6ycmjnW3EXqYG+kPKXmtBZ0s -ZPzAqyMYdXwB6OZ9VDO5KmZH6tpnAGrCVu0kSXDLJxEctXWpaMKMEWZ+nc3saiayaPHUKueQNrEn -zIBwanLjLpLQtBYONkugYFZroSMnISB+Ruqv0gJlOxaS62PZcsNdbxmkAGM1qPU5UXn9AgMBAAEC -gYAs5fiqeteLv1wjBW1HSP1U1B2d2kfIr3t9XORd599vTWszmCK6No8U9dalR0+R6b2l56JGBoBv -EL8joDP2sj2zRR/O1yybvJhWGts/7efHb5T8eraDus+btRlOlwJTfYaLFVmf+eNauaqydgNiv858 -TYfq7hXmYlDRtN11WuvSuQJBAOf/SprCoutduU2Z+YDMmN+t8v0k8DXxSdyyonKhAr85g8p6W6/c -+RylLouRROwqQvIfxnRvbZf68pvKib/zB0MCQQC1qDBouQdsR+K8KXer7kTJBIMccuYotMhm84Yc -3MRHZvR3tfXTJlyFQw3f8FT4sKChmWQA2Rv1sIku1aa8LkW/AkEAhrV/9ljqDMzBGkQ70gP7CHaS -z1axfED86H95QCIp3CtjUSC3OGiQ5YcsfbH+WTs44Cp/K4DAHC2Ezd1PGvvczQJAfesbwzuevZwG -us8zuOXsoDqUd3/dspG96D8d05vkImpnQyPKRpbswyL71yLMg6ZLcjjVbRERpHi+XsySSxb4GQJA -NUK1DEc615gggvYWgSYGjj7XxhrDhhxifukB3eCEWavaDJD8iXQTcmL+64/lTH24Yq10cmDm+d24 -T0aFR1RoPQ== \ No newline at end of file diff --git a/config/keys/privatekey1017 b/config/keys/privatekey1017 deleted file mode 100644 index 4b498b83f..000000000 --- a/config/keys/privatekey1017 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAJTlQg0vlwe6dPOffFxPjTk4fubI -wu/Gw6cxckSU9cXP26qBcof57iJyBI+0tQVGVOSarXFFY1AstYNq/DmhBJLRgtVoTFTheuZqgsdT -QT5uBEdWij/fj3R4jlCqZ3T4N2jwJnGK1JVoM6b8WGC+Fu8Wyd6qMnNirCSPy/g0NdytAgMBAAEC -gYA0okJSJPOPsNLs2AD/JMP4aUc6z9dRKYM6VBlfswSac5rkUDQraydKF+IS5r9i6PTThvYu52dv -DQ7xMN2OhaP2tC/U4HM6Bz5jw4inlpUoHolmVaTp3isqo6XMj2vMLkTeLOE9H/l/fp7QaZ383p6+ -otFavgp4cXKuTKv2/buqhQJBANC/NS3i73w3ypswycGfL6pHAcZTUx5nIuJ4ue5jVL0ZX4MTMZVw -cWv9u+ZYp2vgZhGtCDFLVBEZhKGdNenc0AsCQQC2mbCOqEQ05AsuHYXWW2fum78xQecvLxg3Nqjm -BmgZx0EzAHbl1i87pXsTipYemg/CiQkAHyy2Xw5lNmX6+mEnAkBNoD0zYieypsCmM19jrEG6bYtg -aA4JyUq0szZZSwDxDSVyhcmzi8vx37pnL2KbACiUvCbGxf9uGHI081RgMif/AkEAk8t/yWfOlRTD -K9c/fcnMLKgpL0BNU7aFniMCv/QNjFfnoNVOZydeYaPGI4H3kdtWyDlMbKnO8emsaCWIk4TW3wJA -BZAI8/aRcyqFAKyui29f3nvyMQ5fgOaJRN4nsiPT+inFiIRmvd6KL/3eBrk0y4d73CKOVET4Bdm1 -xGh1TAE27Q== \ No newline at end of file diff --git a/config/keys/privatekey1018 b/config/keys/privatekey1018 deleted file mode 100644 index 4b498b83f..000000000 --- a/config/keys/privatekey1018 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAJTlQg0vlwe6dPOffFxPjTk4fubI -wu/Gw6cxckSU9cXP26qBcof57iJyBI+0tQVGVOSarXFFY1AstYNq/DmhBJLRgtVoTFTheuZqgsdT -QT5uBEdWij/fj3R4jlCqZ3T4N2jwJnGK1JVoM6b8WGC+Fu8Wyd6qMnNirCSPy/g0NdytAgMBAAEC -gYA0okJSJPOPsNLs2AD/JMP4aUc6z9dRKYM6VBlfswSac5rkUDQraydKF+IS5r9i6PTThvYu52dv -DQ7xMN2OhaP2tC/U4HM6Bz5jw4inlpUoHolmVaTp3isqo6XMj2vMLkTeLOE9H/l/fp7QaZ383p6+ -otFavgp4cXKuTKv2/buqhQJBANC/NS3i73w3ypswycGfL6pHAcZTUx5nIuJ4ue5jVL0ZX4MTMZVw -cWv9u+ZYp2vgZhGtCDFLVBEZhKGdNenc0AsCQQC2mbCOqEQ05AsuHYXWW2fum78xQecvLxg3Nqjm -BmgZx0EzAHbl1i87pXsTipYemg/CiQkAHyy2Xw5lNmX6+mEnAkBNoD0zYieypsCmM19jrEG6bYtg -aA4JyUq0szZZSwDxDSVyhcmzi8vx37pnL2KbACiUvCbGxf9uGHI081RgMif/AkEAk8t/yWfOlRTD -K9c/fcnMLKgpL0BNU7aFniMCv/QNjFfnoNVOZydeYaPGI4H3kdtWyDlMbKnO8emsaCWIk4TW3wJA -BZAI8/aRcyqFAKyui29f3nvyMQ5fgOaJRN4nsiPT+inFiIRmvd6KL/3eBrk0y4d73CKOVET4Bdm1 -xGh1TAE27Q== \ No newline at end of file diff --git a/config/keys/privatekey1019 b/config/keys/privatekey1019 deleted file mode 100644 index 6971acfac..000000000 --- a/config/keys/privatekey1019 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBALcyCISI8Yxfdb86aPjyLCIU4qlK -loZ/uWp9W9BiXCSUGkPrgLtdF6j24siUL9V5d6TGmKuUA11guKsUTwpQJUcEGP7LJw80B5sViiXG -DHlnBkGkolsyupKAsajlo/rNY2dzWwMiskwgCf13DWBiExV13B1gRtQt/EEWYV63PgfjAgMBAAEC -gYAmMHnyFxIzrxq+uJB1hRaKLExr+i6S70FZ37QqJrcKsQ0gp5F4fZmXEZCJO8sU93NnmDbvXbWt -26HjmCI6Bice4jPjdB5ZkfZ5dpwqJuuOp0mtivfVjk8hrW3LjagnLOHYzTcIrgveU6L5M3oG6e91 -ilLDF1nJ81cfnM5XSVA7oQJBAOtJ1VcGCZgeMCNfCdC/KViWPm7AaSqMr67BuZpePMWYoi5Sm3pg -PVUV89sDd1GcGlWb31gmFha0udNlC+DbGDsCQQDHUkwxpl48AmM+Nu/6cRKxJy5CL7x5y1Lmok2Y -Ldq3wt54pAITpGLS37WxHWQGQ6X77kr1fFj1fkNkwKjjqXx5AkEAs/Ix0W5yo/5iforO7PFmY7P0 -Z6uMQth5/EN4+ZCdFdoYHylac4trHMm/SH8sICQtF9AwTjBCsHbjr8MWBJBLxQJAUmu5jh5ZgT8L -gsJ84Xa4YADvo/bRXRuyoCQlKfRsKOivzfsj0LN9Ge3f1+lrXxvCGzciT/inYHxhqdvw+pMvIQJA -FVo6Q4VYj1jDe9zKmxL91WUwt85/dvSNcCayNaT5moC4GRFIkWwojsSbN0SCyMhti6u38OCZy1dd -NOhf3eXFXQ== \ No newline at end of file diff --git a/config/keys/privatekey1020 b/config/keys/privatekey1020 deleted file mode 100644 index 6971acfac..000000000 --- a/config/keys/privatekey1020 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBALcyCISI8Yxfdb86aPjyLCIU4qlK -loZ/uWp9W9BiXCSUGkPrgLtdF6j24siUL9V5d6TGmKuUA11guKsUTwpQJUcEGP7LJw80B5sViiXG -DHlnBkGkolsyupKAsajlo/rNY2dzWwMiskwgCf13DWBiExV13B1gRtQt/EEWYV63PgfjAgMBAAEC -gYAmMHnyFxIzrxq+uJB1hRaKLExr+i6S70FZ37QqJrcKsQ0gp5F4fZmXEZCJO8sU93NnmDbvXbWt -26HjmCI6Bice4jPjdB5ZkfZ5dpwqJuuOp0mtivfVjk8hrW3LjagnLOHYzTcIrgveU6L5M3oG6e91 -ilLDF1nJ81cfnM5XSVA7oQJBAOtJ1VcGCZgeMCNfCdC/KViWPm7AaSqMr67BuZpePMWYoi5Sm3pg -PVUV89sDd1GcGlWb31gmFha0udNlC+DbGDsCQQDHUkwxpl48AmM+Nu/6cRKxJy5CL7x5y1Lmok2Y -Ldq3wt54pAITpGLS37WxHWQGQ6X77kr1fFj1fkNkwKjjqXx5AkEAs/Ix0W5yo/5iforO7PFmY7P0 -Z6uMQth5/EN4+ZCdFdoYHylac4trHMm/SH8sICQtF9AwTjBCsHbjr8MWBJBLxQJAUmu5jh5ZgT8L -gsJ84Xa4YADvo/bRXRuyoCQlKfRsKOivzfsj0LN9Ge3f1+lrXxvCGzciT/inYHxhqdvw+pMvIQJA -FVo6Q4VYj1jDe9zKmxL91WUwt85/dvSNcCayNaT5moC4GRFIkWwojsSbN0SCyMhti6u38OCZy1dd -NOhf3eXFXQ== \ No newline at end of file diff --git a/config/keys/privatekey4 b/config/keys/privatekey4 deleted file mode 100644 index 2f4dddad5..000000000 --- a/config/keys/privatekey4 +++ /dev/null @@ -1,12 +0,0 @@ -MIICeAIBADANBgkqhkiG9w0BAQEFAASCAmIwggJeAgEAAoGBAJ7upOa6bxlZTEOdPu/805aJXoZo -EEQPl8imLK9e5X2mNNBo48zd0qsADeVVK5q5Q5bWDE/9QR4e9i11X0G3NIYQsVkrFdCR1O11tRYT -/umhLWVQzfeM+lIY7NP9RMGmedYGcxv0KNRmXAOgIF5bFFXmKCcg0Njmk+FjIHZzdQJrAgMBAAEC -gYEAg4k/MTZbfL9LPyqFTFoTRCOs99BopevB0260zO/dP4CvdU1aO9Pa52vEFGnJDOUH6258Aje4 -DRXS6QybKehWrtBUGT61PccoCIJbiyPnssoR5VlZ2IZSezhzJ0aPEod1/restXf9OzUKQJCsyiwX -WiLB0BqxJailkX6p0bzdlQECQQDQQHb38BtN6o87EuUcO1cnQ3WUsGhO7bKE+wuk3qL1Y1nwbPXp -xSmv8V42gydicMjCHYWG27dVYFCKmfZV46RVAkEAw19Rkq62CZbsmoUhHK2QuyNBipSjyDEqeeYn -hoJlRMHgm++VLaSFK5eNRXRPK4eUuF/gJTQcKMj27HvZzBbLvwJBAMtEINsgoYYLzIH8j2RfB1Jr -pfjbtvRTTkw+Pb9PEAA0WTUzJVRn/aieDzdmDKspKstNxWc8TlYnDHY8kRSEfYECQHgEHvUvlC3x -46jMbVnTmVwMPV/nDXGWGlTFuLQdeB4zx0mjdI2USiZMNAFdgKDp6ccVRnk0OLQpN95v+owv2LsC -QQDCU7qSYzss/HsN1cu+NmDBYW7P60VVYbLqnZYyqs4157N/YRAR2QsAk+0stzkUaTKNcFHurzP1 -a0TcT5mnA6z0 \ No newline at end of file diff --git a/config/keys/privatekey5 b/config/keys/privatekey5 deleted file mode 100644 index d5e66554e..000000000 --- a/config/keys/privatekey5 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKSf6ycmjnW3EXqYG+kPKXmtBZ0s -ZPzAqyMYdXwB6OZ9VDO5KmZH6tpnAGrCVu0kSXDLJxEctXWpaMKMEWZ+nc3saiayaPHUKueQNrEn -zIBwanLjLpLQtBYONkugYFZroSMnISB+Ruqv0gJlOxaS62PZcsNdbxmkAGM1qPU5UXn9AgMBAAEC -gYAs5fiqeteLv1wjBW1HSP1U1B2d2kfIr3t9XORd599vTWszmCK6No8U9dalR0+R6b2l56JGBoBv -EL8joDP2sj2zRR/O1yybvJhWGts/7efHb5T8eraDus+btRlOlwJTfYaLFVmf+eNauaqydgNiv858 -TYfq7hXmYlDRtN11WuvSuQJBAOf/SprCoutduU2Z+YDMmN+t8v0k8DXxSdyyonKhAr85g8p6W6/c -+RylLouRROwqQvIfxnRvbZf68pvKib/zB0MCQQC1qDBouQdsR+K8KXer7kTJBIMccuYotMhm84Yc -3MRHZvR3tfXTJlyFQw3f8FT4sKChmWQA2Rv1sIku1aa8LkW/AkEAhrV/9ljqDMzBGkQ70gP7CHaS -z1axfED86H95QCIp3CtjUSC3OGiQ5YcsfbH+WTs44Cp/K4DAHC2Ezd1PGvvczQJAfesbwzuevZwG -us8zuOXsoDqUd3/dspG96D8d05vkImpnQyPKRpbswyL71yLMg6ZLcjjVbRERpHi+XsySSxb4GQJA -NUK1DEc615gggvYWgSYGjj7XxhrDhhxifukB3eCEWavaDJD8iXQTcmL+64/lTH24Yq10cmDm+d24 -T0aFR1RoPQ== \ No newline at end of file diff --git a/config/keys/privatekey5001 b/config/keys/privatekey5001 deleted file mode 100644 index d5e66554e..000000000 --- a/config/keys/privatekey5001 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKSf6ycmjnW3EXqYG+kPKXmtBZ0s -ZPzAqyMYdXwB6OZ9VDO5KmZH6tpnAGrCVu0kSXDLJxEctXWpaMKMEWZ+nc3saiayaPHUKueQNrEn -zIBwanLjLpLQtBYONkugYFZroSMnISB+Ruqv0gJlOxaS62PZcsNdbxmkAGM1qPU5UXn9AgMBAAEC -gYAs5fiqeteLv1wjBW1HSP1U1B2d2kfIr3t9XORd599vTWszmCK6No8U9dalR0+R6b2l56JGBoBv -EL8joDP2sj2zRR/O1yybvJhWGts/7efHb5T8eraDus+btRlOlwJTfYaLFVmf+eNauaqydgNiv858 -TYfq7hXmYlDRtN11WuvSuQJBAOf/SprCoutduU2Z+YDMmN+t8v0k8DXxSdyyonKhAr85g8p6W6/c -+RylLouRROwqQvIfxnRvbZf68pvKib/zB0MCQQC1qDBouQdsR+K8KXer7kTJBIMccuYotMhm84Yc -3MRHZvR3tfXTJlyFQw3f8FT4sKChmWQA2Rv1sIku1aa8LkW/AkEAhrV/9ljqDMzBGkQ70gP7CHaS -z1axfED86H95QCIp3CtjUSC3OGiQ5YcsfbH+WTs44Cp/K4DAHC2Ezd1PGvvczQJAfesbwzuevZwG -us8zuOXsoDqUd3/dspG96D8d05vkImpnQyPKRpbswyL71yLMg6ZLcjjVbRERpHi+XsySSxb4GQJA -NUK1DEc615gggvYWgSYGjj7XxhrDhhxifukB3eCEWavaDJD8iXQTcmL+64/lTH24Yq10cmDm+d24 -T0aFR1RoPQ== \ No newline at end of file diff --git a/config/keys/privatekey5002 b/config/keys/privatekey5002 deleted file mode 100644 index d5e66554e..000000000 --- a/config/keys/privatekey5002 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAKSf6ycmjnW3EXqYG+kPKXmtBZ0s -ZPzAqyMYdXwB6OZ9VDO5KmZH6tpnAGrCVu0kSXDLJxEctXWpaMKMEWZ+nc3saiayaPHUKueQNrEn -zIBwanLjLpLQtBYONkugYFZroSMnISB+Ruqv0gJlOxaS62PZcsNdbxmkAGM1qPU5UXn9AgMBAAEC -gYAs5fiqeteLv1wjBW1HSP1U1B2d2kfIr3t9XORd599vTWszmCK6No8U9dalR0+R6b2l56JGBoBv -EL8joDP2sj2zRR/O1yybvJhWGts/7efHb5T8eraDus+btRlOlwJTfYaLFVmf+eNauaqydgNiv858 -TYfq7hXmYlDRtN11WuvSuQJBAOf/SprCoutduU2Z+YDMmN+t8v0k8DXxSdyyonKhAr85g8p6W6/c -+RylLouRROwqQvIfxnRvbZf68pvKib/zB0MCQQC1qDBouQdsR+K8KXer7kTJBIMccuYotMhm84Yc -3MRHZvR3tfXTJlyFQw3f8FT4sKChmWQA2Rv1sIku1aa8LkW/AkEAhrV/9ljqDMzBGkQ70gP7CHaS -z1axfED86H95QCIp3CtjUSC3OGiQ5YcsfbH+WTs44Cp/K4DAHC2Ezd1PGvvczQJAfesbwzuevZwG -us8zuOXsoDqUd3/dspG96D8d05vkImpnQyPKRpbswyL71yLMg6ZLcjjVbRERpHi+XsySSxb4GQJA -NUK1DEc615gggvYWgSYGjj7XxhrDhhxifukB3eCEWavaDJD8iXQTcmL+64/lTH24Yq10cmDm+d24 -T0aFR1RoPQ== \ No newline at end of file diff --git a/config/keys/privatekey6 b/config/keys/privatekey6 deleted file mode 100644 index 4b498b83f..000000000 --- a/config/keys/privatekey6 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAJTlQg0vlwe6dPOffFxPjTk4fubI -wu/Gw6cxckSU9cXP26qBcof57iJyBI+0tQVGVOSarXFFY1AstYNq/DmhBJLRgtVoTFTheuZqgsdT -QT5uBEdWij/fj3R4jlCqZ3T4N2jwJnGK1JVoM6b8WGC+Fu8Wyd6qMnNirCSPy/g0NdytAgMBAAEC -gYA0okJSJPOPsNLs2AD/JMP4aUc6z9dRKYM6VBlfswSac5rkUDQraydKF+IS5r9i6PTThvYu52dv -DQ7xMN2OhaP2tC/U4HM6Bz5jw4inlpUoHolmVaTp3isqo6XMj2vMLkTeLOE9H/l/fp7QaZ383p6+ -otFavgp4cXKuTKv2/buqhQJBANC/NS3i73w3ypswycGfL6pHAcZTUx5nIuJ4ue5jVL0ZX4MTMZVw -cWv9u+ZYp2vgZhGtCDFLVBEZhKGdNenc0AsCQQC2mbCOqEQ05AsuHYXWW2fum78xQecvLxg3Nqjm -BmgZx0EzAHbl1i87pXsTipYemg/CiQkAHyy2Xw5lNmX6+mEnAkBNoD0zYieypsCmM19jrEG6bYtg -aA4JyUq0szZZSwDxDSVyhcmzi8vx37pnL2KbACiUvCbGxf9uGHI081RgMif/AkEAk8t/yWfOlRTD -K9c/fcnMLKgpL0BNU7aFniMCv/QNjFfnoNVOZydeYaPGI4H3kdtWyDlMbKnO8emsaCWIk4TW3wJA -BZAI8/aRcyqFAKyui29f3nvyMQ5fgOaJRN4nsiPT+inFiIRmvd6KL/3eBrk0y4d73CKOVET4Bdm1 -xGh1TAE27Q== \ No newline at end of file diff --git a/config/keys/privatekey6001 b/config/keys/privatekey6001 deleted file mode 100644 index 4b498b83f..000000000 --- a/config/keys/privatekey6001 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAJTlQg0vlwe6dPOffFxPjTk4fubI -wu/Gw6cxckSU9cXP26qBcof57iJyBI+0tQVGVOSarXFFY1AstYNq/DmhBJLRgtVoTFTheuZqgsdT -QT5uBEdWij/fj3R4jlCqZ3T4N2jwJnGK1JVoM6b8WGC+Fu8Wyd6qMnNirCSPy/g0NdytAgMBAAEC -gYA0okJSJPOPsNLs2AD/JMP4aUc6z9dRKYM6VBlfswSac5rkUDQraydKF+IS5r9i6PTThvYu52dv -DQ7xMN2OhaP2tC/U4HM6Bz5jw4inlpUoHolmVaTp3isqo6XMj2vMLkTeLOE9H/l/fp7QaZ383p6+ -otFavgp4cXKuTKv2/buqhQJBANC/NS3i73w3ypswycGfL6pHAcZTUx5nIuJ4ue5jVL0ZX4MTMZVw -cWv9u+ZYp2vgZhGtCDFLVBEZhKGdNenc0AsCQQC2mbCOqEQ05AsuHYXWW2fum78xQecvLxg3Nqjm -BmgZx0EzAHbl1i87pXsTipYemg/CiQkAHyy2Xw5lNmX6+mEnAkBNoD0zYieypsCmM19jrEG6bYtg -aA4JyUq0szZZSwDxDSVyhcmzi8vx37pnL2KbACiUvCbGxf9uGHI081RgMif/AkEAk8t/yWfOlRTD -K9c/fcnMLKgpL0BNU7aFniMCv/QNjFfnoNVOZydeYaPGI4H3kdtWyDlMbKnO8emsaCWIk4TW3wJA -BZAI8/aRcyqFAKyui29f3nvyMQ5fgOaJRN4nsiPT+inFiIRmvd6KL/3eBrk0y4d73CKOVET4Bdm1 -xGh1TAE27Q== \ No newline at end of file diff --git a/config/keys/privatekey6002 b/config/keys/privatekey6002 deleted file mode 100644 index 4b498b83f..000000000 --- a/config/keys/privatekey6002 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAJTlQg0vlwe6dPOffFxPjTk4fubI -wu/Gw6cxckSU9cXP26qBcof57iJyBI+0tQVGVOSarXFFY1AstYNq/DmhBJLRgtVoTFTheuZqgsdT -QT5uBEdWij/fj3R4jlCqZ3T4N2jwJnGK1JVoM6b8WGC+Fu8Wyd6qMnNirCSPy/g0NdytAgMBAAEC -gYA0okJSJPOPsNLs2AD/JMP4aUc6z9dRKYM6VBlfswSac5rkUDQraydKF+IS5r9i6PTThvYu52dv -DQ7xMN2OhaP2tC/U4HM6Bz5jw4inlpUoHolmVaTp3isqo6XMj2vMLkTeLOE9H/l/fp7QaZ383p6+ -otFavgp4cXKuTKv2/buqhQJBANC/NS3i73w3ypswycGfL6pHAcZTUx5nIuJ4ue5jVL0ZX4MTMZVw -cWv9u+ZYp2vgZhGtCDFLVBEZhKGdNenc0AsCQQC2mbCOqEQ05AsuHYXWW2fum78xQecvLxg3Nqjm -BmgZx0EzAHbl1i87pXsTipYemg/CiQkAHyy2Xw5lNmX6+mEnAkBNoD0zYieypsCmM19jrEG6bYtg -aA4JyUq0szZZSwDxDSVyhcmzi8vx37pnL2KbACiUvCbGxf9uGHI081RgMif/AkEAk8t/yWfOlRTD -K9c/fcnMLKgpL0BNU7aFniMCv/QNjFfnoNVOZydeYaPGI4H3kdtWyDlMbKnO8emsaCWIk4TW3wJA -BZAI8/aRcyqFAKyui29f3nvyMQ5fgOaJRN4nsiPT+inFiIRmvd6KL/3eBrk0y4d73CKOVET4Bdm1 -xGh1TAE27Q== \ No newline at end of file diff --git a/config/keys/privatekey7 b/config/keys/privatekey7 deleted file mode 100644 index 6971acfac..000000000 --- a/config/keys/privatekey7 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBALcyCISI8Yxfdb86aPjyLCIU4qlK -loZ/uWp9W9BiXCSUGkPrgLtdF6j24siUL9V5d6TGmKuUA11guKsUTwpQJUcEGP7LJw80B5sViiXG -DHlnBkGkolsyupKAsajlo/rNY2dzWwMiskwgCf13DWBiExV13B1gRtQt/EEWYV63PgfjAgMBAAEC -gYAmMHnyFxIzrxq+uJB1hRaKLExr+i6S70FZ37QqJrcKsQ0gp5F4fZmXEZCJO8sU93NnmDbvXbWt -26HjmCI6Bice4jPjdB5ZkfZ5dpwqJuuOp0mtivfVjk8hrW3LjagnLOHYzTcIrgveU6L5M3oG6e91 -ilLDF1nJ81cfnM5XSVA7oQJBAOtJ1VcGCZgeMCNfCdC/KViWPm7AaSqMr67BuZpePMWYoi5Sm3pg -PVUV89sDd1GcGlWb31gmFha0udNlC+DbGDsCQQDHUkwxpl48AmM+Nu/6cRKxJy5CL7x5y1Lmok2Y -Ldq3wt54pAITpGLS37WxHWQGQ6X77kr1fFj1fkNkwKjjqXx5AkEAs/Ix0W5yo/5iforO7PFmY7P0 -Z6uMQth5/EN4+ZCdFdoYHylac4trHMm/SH8sICQtF9AwTjBCsHbjr8MWBJBLxQJAUmu5jh5ZgT8L -gsJ84Xa4YADvo/bRXRuyoCQlKfRsKOivzfsj0LN9Ge3f1+lrXxvCGzciT/inYHxhqdvw+pMvIQJA -FVo6Q4VYj1jDe9zKmxL91WUwt85/dvSNcCayNaT5moC4GRFIkWwojsSbN0SCyMhti6u38OCZy1dd -NOhf3eXFXQ== \ No newline at end of file diff --git a/config/keys/privatekey7001 b/config/keys/privatekey7001 deleted file mode 100644 index 6971acfac..000000000 --- a/config/keys/privatekey7001 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBALcyCISI8Yxfdb86aPjyLCIU4qlK -loZ/uWp9W9BiXCSUGkPrgLtdF6j24siUL9V5d6TGmKuUA11guKsUTwpQJUcEGP7LJw80B5sViiXG -DHlnBkGkolsyupKAsajlo/rNY2dzWwMiskwgCf13DWBiExV13B1gRtQt/EEWYV63PgfjAgMBAAEC -gYAmMHnyFxIzrxq+uJB1hRaKLExr+i6S70FZ37QqJrcKsQ0gp5F4fZmXEZCJO8sU93NnmDbvXbWt -26HjmCI6Bice4jPjdB5ZkfZ5dpwqJuuOp0mtivfVjk8hrW3LjagnLOHYzTcIrgveU6L5M3oG6e91 -ilLDF1nJ81cfnM5XSVA7oQJBAOtJ1VcGCZgeMCNfCdC/KViWPm7AaSqMr67BuZpePMWYoi5Sm3pg -PVUV89sDd1GcGlWb31gmFha0udNlC+DbGDsCQQDHUkwxpl48AmM+Nu/6cRKxJy5CL7x5y1Lmok2Y -Ldq3wt54pAITpGLS37WxHWQGQ6X77kr1fFj1fkNkwKjjqXx5AkEAs/Ix0W5yo/5iforO7PFmY7P0 -Z6uMQth5/EN4+ZCdFdoYHylac4trHMm/SH8sICQtF9AwTjBCsHbjr8MWBJBLxQJAUmu5jh5ZgT8L -gsJ84Xa4YADvo/bRXRuyoCQlKfRsKOivzfsj0LN9Ge3f1+lrXxvCGzciT/inYHxhqdvw+pMvIQJA -FVo6Q4VYj1jDe9zKmxL91WUwt85/dvSNcCayNaT5moC4GRFIkWwojsSbN0SCyMhti6u38OCZy1dd -NOhf3eXFXQ== \ No newline at end of file diff --git a/config/keys/privatekey7002 b/config/keys/privatekey7002 deleted file mode 100644 index 6971acfac..000000000 --- a/config/keys/privatekey7002 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBALcyCISI8Yxfdb86aPjyLCIU4qlK -loZ/uWp9W9BiXCSUGkPrgLtdF6j24siUL9V5d6TGmKuUA11guKsUTwpQJUcEGP7LJw80B5sViiXG -DHlnBkGkolsyupKAsajlo/rNY2dzWwMiskwgCf13DWBiExV13B1gRtQt/EEWYV63PgfjAgMBAAEC -gYAmMHnyFxIzrxq+uJB1hRaKLExr+i6S70FZ37QqJrcKsQ0gp5F4fZmXEZCJO8sU93NnmDbvXbWt -26HjmCI6Bice4jPjdB5ZkfZ5dpwqJuuOp0mtivfVjk8hrW3LjagnLOHYzTcIrgveU6L5M3oG6e91 -ilLDF1nJ81cfnM5XSVA7oQJBAOtJ1VcGCZgeMCNfCdC/KViWPm7AaSqMr67BuZpePMWYoi5Sm3pg -PVUV89sDd1GcGlWb31gmFha0udNlC+DbGDsCQQDHUkwxpl48AmM+Nu/6cRKxJy5CL7x5y1Lmok2Y -Ldq3wt54pAITpGLS37WxHWQGQ6X77kr1fFj1fkNkwKjjqXx5AkEAs/Ix0W5yo/5iforO7PFmY7P0 -Z6uMQth5/EN4+ZCdFdoYHylac4trHMm/SH8sICQtF9AwTjBCsHbjr8MWBJBLxQJAUmu5jh5ZgT8L -gsJ84Xa4YADvo/bRXRuyoCQlKfRsKOivzfsj0LN9Ge3f1+lrXxvCGzciT/inYHxhqdvw+pMvIQJA -FVo6Q4VYj1jDe9zKmxL91WUwt85/dvSNcCayNaT5moC4GRFIkWwojsSbN0SCyMhti6u38OCZy1dd -NOhf3eXFXQ== \ No newline at end of file diff --git a/config/keys/privatekey8 b/config/keys/privatekey8 deleted file mode 100644 index 21f1ce652..000000000 --- a/config/keys/privatekey8 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAJsjCfOuDPHVrKNCdYBNk2uMS3bA -o+A9ShNthw1M38UeM8v5NMZJW3xOH9/HqX8XPDmzVmpYWvPMwX8rjTqYbBh72kuPxzR6URhEtrGs -Iaru+woH/Q4pgP3k+VyoLWxnB6rFb/AEThr1BWKUMeISfjEjPRY9qqptnw9SBTr7/WWnAgMBAAEC -gYAxhgDySjiVpslioJ+T4q/R4Epi0WI+/jG8e/Vl2ImCin6seZRQTDCpQONJ85n3GTq/jLFkZ2o5 -c769UvDVqLywoFK0H0wCu1T+atHtBwhjmmE1okkACato/BxUiySGYiTagJOkLBTkWdPPE4i9aZqm -EsSEsJFV4c8JiudL8H8ZKQJBANwqlAKaEmxiNsEinfT8R8mCLQc7GaX5Rud2jxbiaGI255/FMfAY -tTsGnN6WKqYbMbVQXVQjxz9COIxUKre3ikUCQQC0YvOxo7RWczWnjLAinTxx891Qp37v7Yn9gdDq -IRZEeqiHB/aaTsVFy5zV1ElaDkBstL82hMobUtmDkUmJPMT7AkEAj4BBEKETuQka3y+M/Q0iTTlH -2BqxUPj1OhCQmz3fA4T926lQL0orddmNjZcbHUFEWQWUjy7yw0yKRwzk+vHqcQJAIyM7URkQYd6j -TvXqwXNbeN8F91YJNfdRb94/sbusOL6j4Vm9waUibEwTStvFRhOMnSBANwB61gO3dxdv+ZmqzQJA -Sb0R/zHeP+6EVCmiFUYqFnUcFh3K1t9yWWEAaU+/9EGnAhi0nee5GzaRbL0tZwCCDxkTYX149NKT -Zscf88xQcQ== \ No newline at end of file diff --git a/config/keys/privatekey9 b/config/keys/privatekey9 deleted file mode 100644 index 2dc2d9874..000000000 --- a/config/keys/privatekey9 +++ /dev/null @@ -1,12 +0,0 @@ -MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBAIOsuw2PdzI2NvWA81iltnmj+6Xn -Un1sMQ8kNro6Z9h0vrEjwccjtqsAw5KupzLFjgZBZlld3xwCiKdvXH0C5Lml9hCB3vSolnb3oWyU -NyS59TvwY0yB/IM+ZYICO+b2LLoo+72DR0HUPbWQOXXAg4QFNCIBVi3G+hI1aO2I3e63AgMBAAEC -gYBv0O3jXRJGhH11S7TacZacg2F+iAqn0YlzpzgGOvTWfoU03f1/Q1eqrXJnDIOVfOfXbhAhEbMW -ERzEuq+axSs/LcjYSlj1PyWbKm0OoiVQS9YklqVl2oQyN7mrUspRBnvq35M33+cOec/0BnbXjl8L -H2N2q5xC/7b76U83ktKJgQJBAOyNtcwcsY0J+NZNJQwdpW4hmOriy672AZjAZ7spSrXngF3rTrqP -N/dIczudLYVc6Ws04aJdHnadlR4TMfqcZUECQQCOf9YuM8pSdqVsM4cS/E8DvmbM1w7OY3HWYuZz -4wsWWylDbiBu0ETsGC2M+ojfd4Ap6/DPayT/g1GBZRn/J/33AkEA6QjJmaSCvQCf5O3mm0LxX8c1 -T9/Q7DFpbUPObLDG6uB+swdGbb79UVxMOOQkngbj5DsaNMsrYYroBdK/H7XGgQJAFuyL2SztvGxn -2ktDzJMYZwLHaOaj9CHdW17XsgbxbqxFrBpYT1tj6xNMqKRZRpHj5VqWh9e1wQEwKH/KOdkGKwJB -AOEO/Xs8BWnzF/vsXGN7E/xiFOhio4mP862NqQSL8cxRmswzMeLJ3cdv1Z1lxXGFIY+aOczB6Sm4 -jnszzWPp0yA= \ No newline at end of file diff --git a/config/keys/publickey1002 b/config/keys/publickey1002 deleted file mode 100644 index ce581a113..000000000 --- a/config/keys/publickey1002 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDrLsNj3cyNjb1gPNYpbZ5o/ul51J9bDEPJDa6 -OmfYdL6xI8HHI7arAMOSrqcyxY4GQWZZXd8cAoinb1x9AuS5pfYQgd70qJZ296FslDckufU78GNM -gfyDPmWCAjvm9iy6KPu9g0dB1D21kDl1wIOEBTQiAVYtxvoSNWjtiN3utwIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1003 b/config/keys/publickey1003 deleted file mode 100644 index ce581a113..000000000 --- a/config/keys/publickey1003 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDrLsNj3cyNjb1gPNYpbZ5o/ul51J9bDEPJDa6 -OmfYdL6xI8HHI7arAMOSrqcyxY4GQWZZXd8cAoinb1x9AuS5pfYQgd70qJZ296FslDckufU78GNM -gfyDPmWCAjvm9iy6KPu9g0dB1D21kDl1wIOEBTQiAVYtxvoSNWjtiN3utwIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1004 b/config/keys/publickey1004 deleted file mode 100644 index ce581a113..000000000 --- a/config/keys/publickey1004 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDrLsNj3cyNjb1gPNYpbZ5o/ul51J9bDEPJDa6 -OmfYdL6xI8HHI7arAMOSrqcyxY4GQWZZXd8cAoinb1x9AuS5pfYQgd70qJZ296FslDckufU78GNM -gfyDPmWCAjvm9iy6KPu9g0dB1D21kDl1wIOEBTQiAVYtxvoSNWjtiN3utwIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1005 b/config/keys/publickey1005 deleted file mode 100644 index e0e1457e0..000000000 --- a/config/keys/publickey1005 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkn+snJo51txF6mBvpDyl5rQWdLGT8wKsjGHV8 -AejmfVQzuSpmR+raZwBqwlbtJElwyycRHLV1qWjCjBFmfp3N7Gomsmjx1CrnkDaxJ8yAcGpy4y6S -0LQWDjZLoGBWa6EjJyEgfkbqr9ICZTsWkutj2XLDXW8ZpABjNaj1OVF5/QIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1006 b/config/keys/publickey1006 deleted file mode 100644 index e0e1457e0..000000000 --- a/config/keys/publickey1006 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkn+snJo51txF6mBvpDyl5rQWdLGT8wKsjGHV8 -AejmfVQzuSpmR+raZwBqwlbtJElwyycRHLV1qWjCjBFmfp3N7Gomsmjx1CrnkDaxJ8yAcGpy4y6S -0LQWDjZLoGBWa6EjJyEgfkbqr9ICZTsWkutj2XLDXW8ZpABjNaj1OVF5/QIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1007 b/config/keys/publickey1007 deleted file mode 100644 index 0d6038381..000000000 --- a/config/keys/publickey1007 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCU5UINL5cHunTzn3xcT405OH7myMLvxsOnMXJE -lPXFz9uqgXKH+e4icgSPtLUFRlTkmq1xRWNQLLWDavw5oQSS0YLVaExU4XrmaoLHU0E+bgRHVoo/ -3490eI5Qqmd0+Ddo8CZxitSVaDOm/FhgvhbvFsneqjJzYqwkj8v4NDXcrQIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1008 b/config/keys/publickey1008 deleted file mode 100644 index 0d6038381..000000000 --- a/config/keys/publickey1008 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCU5UINL5cHunTzn3xcT405OH7myMLvxsOnMXJE -lPXFz9uqgXKH+e4icgSPtLUFRlTkmq1xRWNQLLWDavw5oQSS0YLVaExU4XrmaoLHU0E+bgRHVoo/ -3490eI5Qqmd0+Ddo8CZxitSVaDOm/FhgvhbvFsneqjJzYqwkj8v4NDXcrQIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1009 b/config/keys/publickey1009 deleted file mode 100644 index 84697208f..000000000 --- a/config/keys/publickey1009 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3MgiEiPGMX3W/Omj48iwiFOKpSpaGf7lqfVvQ -YlwklBpD64C7XReo9uLIlC/VeXekxpirlANdYLirFE8KUCVHBBj+yycPNAebFYolxgx5ZwZBpKJb -MrqSgLGo5aP6zWNnc1sDIrJMIAn9dw1gYhMVddwdYEbULfxBFmFetz4H4wIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1010 b/config/keys/publickey1010 deleted file mode 100644 index 84697208f..000000000 --- a/config/keys/publickey1010 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3MgiEiPGMX3W/Omj48iwiFOKpSpaGf7lqfVvQ -YlwklBpD64C7XReo9uLIlC/VeXekxpirlANdYLirFE8KUCVHBBj+yycPNAebFYolxgx5ZwZBpKJb -MrqSgLGo5aP6zWNnc1sDIrJMIAn9dw1gYhMVddwdYEbULfxBFmFetz4H4wIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1011 b/config/keys/publickey1011 deleted file mode 100644 index ce581a113..000000000 --- a/config/keys/publickey1011 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDrLsNj3cyNjb1gPNYpbZ5o/ul51J9bDEPJDa6 -OmfYdL6xI8HHI7arAMOSrqcyxY4GQWZZXd8cAoinb1x9AuS5pfYQgd70qJZ296FslDckufU78GNM -gfyDPmWCAjvm9iy6KPu9g0dB1D21kDl1wIOEBTQiAVYtxvoSNWjtiN3utwIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1012 b/config/keys/publickey1012 deleted file mode 100644 index ce581a113..000000000 --- a/config/keys/publickey1012 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDrLsNj3cyNjb1gPNYpbZ5o/ul51J9bDEPJDa6 -OmfYdL6xI8HHI7arAMOSrqcyxY4GQWZZXd8cAoinb1x9AuS5pfYQgd70qJZ296FslDckufU78GNM -gfyDPmWCAjvm9iy6KPu9g0dB1D21kDl1wIOEBTQiAVYtxvoSNWjtiN3utwIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1013 b/config/keys/publickey1013 deleted file mode 100644 index ce581a113..000000000 --- a/config/keys/publickey1013 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDrLsNj3cyNjb1gPNYpbZ5o/ul51J9bDEPJDa6 -OmfYdL6xI8HHI7arAMOSrqcyxY4GQWZZXd8cAoinb1x9AuS5pfYQgd70qJZ296FslDckufU78GNM -gfyDPmWCAjvm9iy6KPu9g0dB1D21kDl1wIOEBTQiAVYtxvoSNWjtiN3utwIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1014 b/config/keys/publickey1014 deleted file mode 100644 index ce581a113..000000000 --- a/config/keys/publickey1014 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDrLsNj3cyNjb1gPNYpbZ5o/ul51J9bDEPJDa6 -OmfYdL6xI8HHI7arAMOSrqcyxY4GQWZZXd8cAoinb1x9AuS5pfYQgd70qJZ296FslDckufU78GNM -gfyDPmWCAjvm9iy6KPu9g0dB1D21kDl1wIOEBTQiAVYtxvoSNWjtiN3utwIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1015 b/config/keys/publickey1015 deleted file mode 100644 index e0e1457e0..000000000 --- a/config/keys/publickey1015 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkn+snJo51txF6mBvpDyl5rQWdLGT8wKsjGHV8 -AejmfVQzuSpmR+raZwBqwlbtJElwyycRHLV1qWjCjBFmfp3N7Gomsmjx1CrnkDaxJ8yAcGpy4y6S -0LQWDjZLoGBWa6EjJyEgfkbqr9ICZTsWkutj2XLDXW8ZpABjNaj1OVF5/QIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1016 b/config/keys/publickey1016 deleted file mode 100644 index e0e1457e0..000000000 --- a/config/keys/publickey1016 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkn+snJo51txF6mBvpDyl5rQWdLGT8wKsjGHV8 -AejmfVQzuSpmR+raZwBqwlbtJElwyycRHLV1qWjCjBFmfp3N7Gomsmjx1CrnkDaxJ8yAcGpy4y6S -0LQWDjZLoGBWa6EjJyEgfkbqr9ICZTsWkutj2XLDXW8ZpABjNaj1OVF5/QIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1017 b/config/keys/publickey1017 deleted file mode 100644 index 0d6038381..000000000 --- a/config/keys/publickey1017 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCU5UINL5cHunTzn3xcT405OH7myMLvxsOnMXJE -lPXFz9uqgXKH+e4icgSPtLUFRlTkmq1xRWNQLLWDavw5oQSS0YLVaExU4XrmaoLHU0E+bgRHVoo/ -3490eI5Qqmd0+Ddo8CZxitSVaDOm/FhgvhbvFsneqjJzYqwkj8v4NDXcrQIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1018 b/config/keys/publickey1018 deleted file mode 100644 index 0d6038381..000000000 --- a/config/keys/publickey1018 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCU5UINL5cHunTzn3xcT405OH7myMLvxsOnMXJE -lPXFz9uqgXKH+e4icgSPtLUFRlTkmq1xRWNQLLWDavw5oQSS0YLVaExU4XrmaoLHU0E+bgRHVoo/ -3490eI5Qqmd0+Ddo8CZxitSVaDOm/FhgvhbvFsneqjJzYqwkj8v4NDXcrQIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1019 b/config/keys/publickey1019 deleted file mode 100644 index 84697208f..000000000 --- a/config/keys/publickey1019 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3MgiEiPGMX3W/Omj48iwiFOKpSpaGf7lqfVvQ -YlwklBpD64C7XReo9uLIlC/VeXekxpirlANdYLirFE8KUCVHBBj+yycPNAebFYolxgx5ZwZBpKJb -MrqSgLGo5aP6zWNnc1sDIrJMIAn9dw1gYhMVddwdYEbULfxBFmFetz4H4wIDAQAB \ No newline at end of file diff --git a/config/keys/publickey1020 b/config/keys/publickey1020 deleted file mode 100644 index 84697208f..000000000 --- a/config/keys/publickey1020 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3MgiEiPGMX3W/Omj48iwiFOKpSpaGf7lqfVvQ -YlwklBpD64C7XReo9uLIlC/VeXekxpirlANdYLirFE8KUCVHBBj+yycPNAebFYolxgx5ZwZBpKJb -MrqSgLGo5aP6zWNnc1sDIrJMIAn9dw1gYhMVddwdYEbULfxBFmFetz4H4wIDAQAB \ No newline at end of file diff --git a/config/keys/publickey4 b/config/keys/publickey4 deleted file mode 100644 index 599915d1a..000000000 --- a/config/keys/publickey4 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCe7qTmum8ZWUxDnT7v/NOWiV6GaBBED5fIpiyv -XuV9pjTQaOPM3dKrAA3lVSuauUOW1gxP/UEeHvYtdV9BtzSGELFZKxXQkdTtdbUWE/7poS1lUM33 -jPpSGOzT/UTBpnnWBnMb9CjUZlwDoCBeWxRV5ignINDY5pPhYyB2c3UCawIDAQAB \ No newline at end of file diff --git a/config/keys/publickey5 b/config/keys/publickey5 deleted file mode 100644 index e0e1457e0..000000000 --- a/config/keys/publickey5 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkn+snJo51txF6mBvpDyl5rQWdLGT8wKsjGHV8 -AejmfVQzuSpmR+raZwBqwlbtJElwyycRHLV1qWjCjBFmfp3N7Gomsmjx1CrnkDaxJ8yAcGpy4y6S -0LQWDjZLoGBWa6EjJyEgfkbqr9ICZTsWkutj2XLDXW8ZpABjNaj1OVF5/QIDAQAB \ No newline at end of file diff --git a/config/keys/publickey5001 b/config/keys/publickey5001 deleted file mode 100644 index e0e1457e0..000000000 --- a/config/keys/publickey5001 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkn+snJo51txF6mBvpDyl5rQWdLGT8wKsjGHV8 -AejmfVQzuSpmR+raZwBqwlbtJElwyycRHLV1qWjCjBFmfp3N7Gomsmjx1CrnkDaxJ8yAcGpy4y6S -0LQWDjZLoGBWa6EjJyEgfkbqr9ICZTsWkutj2XLDXW8ZpABjNaj1OVF5/QIDAQAB \ No newline at end of file diff --git a/config/keys/publickey5002 b/config/keys/publickey5002 deleted file mode 100644 index e0e1457e0..000000000 --- a/config/keys/publickey5002 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCkn+snJo51txF6mBvpDyl5rQWdLGT8wKsjGHV8 -AejmfVQzuSpmR+raZwBqwlbtJElwyycRHLV1qWjCjBFmfp3N7Gomsmjx1CrnkDaxJ8yAcGpy4y6S -0LQWDjZLoGBWa6EjJyEgfkbqr9ICZTsWkutj2XLDXW8ZpABjNaj1OVF5/QIDAQAB \ No newline at end of file diff --git a/config/keys/publickey6 b/config/keys/publickey6 deleted file mode 100644 index 0d6038381..000000000 --- a/config/keys/publickey6 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCU5UINL5cHunTzn3xcT405OH7myMLvxsOnMXJE -lPXFz9uqgXKH+e4icgSPtLUFRlTkmq1xRWNQLLWDavw5oQSS0YLVaExU4XrmaoLHU0E+bgRHVoo/ -3490eI5Qqmd0+Ddo8CZxitSVaDOm/FhgvhbvFsneqjJzYqwkj8v4NDXcrQIDAQAB \ No newline at end of file diff --git a/config/keys/publickey6001 b/config/keys/publickey6001 deleted file mode 100644 index 0d6038381..000000000 --- a/config/keys/publickey6001 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCU5UINL5cHunTzn3xcT405OH7myMLvxsOnMXJE -lPXFz9uqgXKH+e4icgSPtLUFRlTkmq1xRWNQLLWDavw5oQSS0YLVaExU4XrmaoLHU0E+bgRHVoo/ -3490eI5Qqmd0+Ddo8CZxitSVaDOm/FhgvhbvFsneqjJzYqwkj8v4NDXcrQIDAQAB \ No newline at end of file diff --git a/config/keys/publickey6002 b/config/keys/publickey6002 deleted file mode 100644 index 0d6038381..000000000 --- a/config/keys/publickey6002 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCU5UINL5cHunTzn3xcT405OH7myMLvxsOnMXJE -lPXFz9uqgXKH+e4icgSPtLUFRlTkmq1xRWNQLLWDavw5oQSS0YLVaExU4XrmaoLHU0E+bgRHVoo/ -3490eI5Qqmd0+Ddo8CZxitSVaDOm/FhgvhbvFsneqjJzYqwkj8v4NDXcrQIDAQAB \ No newline at end of file diff --git a/config/keys/publickey7 b/config/keys/publickey7 deleted file mode 100644 index 84697208f..000000000 --- a/config/keys/publickey7 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3MgiEiPGMX3W/Omj48iwiFOKpSpaGf7lqfVvQ -YlwklBpD64C7XReo9uLIlC/VeXekxpirlANdYLirFE8KUCVHBBj+yycPNAebFYolxgx5ZwZBpKJb -MrqSgLGo5aP6zWNnc1sDIrJMIAn9dw1gYhMVddwdYEbULfxBFmFetz4H4wIDAQAB \ No newline at end of file diff --git a/config/keys/publickey7001 b/config/keys/publickey7001 deleted file mode 100644 index 84697208f..000000000 --- a/config/keys/publickey7001 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3MgiEiPGMX3W/Omj48iwiFOKpSpaGf7lqfVvQ -YlwklBpD64C7XReo9uLIlC/VeXekxpirlANdYLirFE8KUCVHBBj+yycPNAebFYolxgx5ZwZBpKJb -MrqSgLGo5aP6zWNnc1sDIrJMIAn9dw1gYhMVddwdYEbULfxBFmFetz4H4wIDAQAB \ No newline at end of file diff --git a/config/keys/publickey7002 b/config/keys/publickey7002 deleted file mode 100644 index 84697208f..000000000 --- a/config/keys/publickey7002 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC3MgiEiPGMX3W/Omj48iwiFOKpSpaGf7lqfVvQ -YlwklBpD64C7XReo9uLIlC/VeXekxpirlANdYLirFE8KUCVHBBj+yycPNAebFYolxgx5ZwZBpKJb -MrqSgLGo5aP6zWNnc1sDIrJMIAn9dw1gYhMVddwdYEbULfxBFmFetz4H4wIDAQAB \ No newline at end of file diff --git a/config/keys/publickey8 b/config/keys/publickey8 deleted file mode 100644 index c2688dd0b..000000000 --- a/config/keys/publickey8 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCbIwnzrgzx1ayjQnWATZNrjEt2wKPgPUoTbYcN -TN/FHjPL+TTGSVt8Th/fx6l/Fzw5s1ZqWFrzzMF/K406mGwYe9pLj8c0elEYRLaxrCGq7vsKB/0O -KYD95PlcqC1sZweqxW/wBE4a9QVilDHiEn4xIz0WPaqqbZ8PUgU6+/1lpwIDAQAB \ No newline at end of file diff --git a/config/keys/publickey9 b/config/keys/publickey9 deleted file mode 100644 index ce581a113..000000000 --- a/config/keys/publickey9 +++ /dev/null @@ -1,3 +0,0 @@ -MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCDrLsNj3cyNjb1gPNYpbZ5o/ul51J9bDEPJDa6 -OmfYdL6xI8HHI7arAMOSrqcyxY4GQWZZXd8cAoinb1x9AuS5pfYQgd70qJZ296FslDckufU78GNM -gfyDPmWCAjvm9iy6KPu9g0dB1D21kDl1wIOEBTQiAVYtxvoSNWjtiN3utwIDAQAB \ No newline at end of file diff --git a/config/keysECDSA/privatekey0 b/config/keysECDSA/privatekey0 new file mode 100644 index 000000000..34cacbf84 --- /dev/null +++ b/config/keysECDSA/privatekey0 @@ -0,0 +1 @@ +MIGTAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQgTAiHxNOFYHLQH5t2QpNcf1kfv5sQkqaje2+ly88YWfWgCgYIKoZIzj0DAQehRANCAATsVazn4VOUZQVvuC263gKrot+jnJLuDALyAMlutD5gV5+QpgxIxSOjlEIbXGsUdoWtVpmmJUQ6iTMhxBedCC3+ \ No newline at end of file diff --git a/config/keysECDSA/privatekey1 b/config/keysECDSA/privatekey1 new file mode 100644 index 000000000..9ac186672 --- /dev/null +++ b/config/keysECDSA/privatekey1 @@ -0,0 +1 @@ +MEECAQAwEwYHKoZIzj0CAQYIKoZIzj0DAQcEJzAlAgEBBCBisAauousSwa53qbwzSnA/H6zWCZCzB+PQnGpGKMUnZA== \ No newline at end of file diff --git a/config/keysECDSA/privatekey1001 b/config/keysECDSA/privatekey1001 new file mode 100644 index 000000000..4ec71f992 --- /dev/null +++ b/config/keysECDSA/privatekey1001 @@ -0,0 +1 @@ +MIGTAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQgG/obXkbU5IFp1es5y07ySarDTtW3ybKSpsSP+ryfcAigCgYIKoZIzj0DAQehRANCAARBLh9grsICxjK8pCjG8S/0bCk+9W9xXmw1u7pkeucn0poGcKHh67zcq0mfsYk2L0Ld9J9KHerYZqb4AiEv/mUV \ No newline at end of file diff --git a/config/keysECDSA/privatekey2 b/config/keysECDSA/privatekey2 new file mode 100644 index 000000000..76245b869 --- /dev/null +++ b/config/keysECDSA/privatekey2 @@ -0,0 +1 @@ +MEECAQAwEwYHKoZIzj0CAQYIKoZIzj0DAQcEJzAlAgEBBCB/eMk4/sUQZUVFQh30Cn0HPdcHOU8wvXb/mbdLwFDq6g== \ No newline at end of file diff --git a/config/keysECDSA/privatekey3 b/config/keysECDSA/privatekey3 new file mode 100644 index 000000000..f96022f92 --- /dev/null +++ b/config/keysECDSA/privatekey3 @@ -0,0 +1 @@ +MIGTAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQga0/il+FZFb4zEY7IYFTXJA/yahL0tuO9D30L7V6aOVCgCgYIKoZIzj0DAQehRANCAAQuQvTd3t56uaXpmHnpN3AJ9RAqXNtw4kgll2j73IY25STuVnK4gWPGhVqukVwc1SQfQLGEaT6+GpDTjiRnb3u4 \ No newline at end of file diff --git a/config/keysECDSA/publickey0 b/config/keysECDSA/publickey0 new file mode 100644 index 000000000..a26942d38 --- /dev/null +++ b/config/keysECDSA/publickey0 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE7FWs5+FTlGUFb7gtut4Cq6Lfo5yS7gwC8gDJbrQ+YFefkKYMSMUjo5RCG1xrFHaFrVaZpiVEOokzIcQXnQgt/g== \ No newline at end of file diff --git a/config/keysECDSA/publickey1 b/config/keysECDSA/publickey1 new file mode 100644 index 000000000..86e356136 --- /dev/null +++ b/config/keysECDSA/publickey1 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEf8n4phNQcxy4dWXVQ+mgohjf1DEWOBo82oTCNqjDa/z6hexXRP4vQ5rV6qjlwkHex8JCwmGkfV9CcmqzgaQx7A== \ No newline at end of file diff --git a/config/keysECDSA/publickey1001 b/config/keysECDSA/publickey1001 new file mode 100644 index 000000000..bd92f302b --- /dev/null +++ b/config/keysECDSA/publickey1001 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEQS4fYK7CAsYyvKQoxvEv9GwpPvVvcV5sNbu6ZHrnJ9KaBnCh4eu83KtJn7GJNi9C3fSfSh3q2Gam+AIhL/5lFQ== \ No newline at end of file diff --git a/config/keysECDSA/publickey2 b/config/keysECDSA/publickey2 new file mode 100644 index 000000000..81c4c6c78 --- /dev/null +++ b/config/keysECDSA/publickey2 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE0aau6c8rDUDi9EBylL/66wcdng+80IHk0xi23cJB9KkAD/lEBxF3Y67a+ewyWLZb7MeTBJ8z9x8vQEZIaj8RSg== \ No newline at end of file diff --git a/config/keysECDSA/publickey3 b/config/keysECDSA/publickey3 new file mode 100644 index 000000000..d67cffba1 --- /dev/null +++ b/config/keysECDSA/publickey3 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAELkL03d7eerml6Zh56TdwCfUQKlzbcOJIJZdo+9yGNuUk7lZyuIFjxoVarpFcHNUkH0CxhGk+vhqQ044kZ297uA== \ No newline at end of file diff --git a/config/keys/privatekey b/config/keysRSA/privatekey similarity index 100% rename from config/keys/privatekey rename to config/keysRSA/privatekey diff --git a/config/keys/privatekey0 b/config/keysRSA/privatekey0 similarity index 100% rename from config/keys/privatekey0 rename to config/keysRSA/privatekey0 diff --git a/config/keys/privatekey1 b/config/keysRSA/privatekey1 similarity index 100% rename from config/keys/privatekey1 rename to config/keysRSA/privatekey1 diff --git a/config/keys/privatekey1001 b/config/keysRSA/privatekey1001 similarity index 100% rename from config/keys/privatekey1001 rename to config/keysRSA/privatekey1001 diff --git a/config/keys/privatekey2 b/config/keysRSA/privatekey2 similarity index 100% rename from config/keys/privatekey2 rename to config/keysRSA/privatekey2 diff --git a/config/keys/privatekey3 b/config/keysRSA/privatekey3 similarity index 100% rename from config/keys/privatekey3 rename to config/keysRSA/privatekey3 diff --git a/config/keys/publickey b/config/keysRSA/publickey similarity index 100% rename from config/keys/publickey rename to config/keysRSA/publickey diff --git a/config/keys/publickey0 b/config/keysRSA/publickey0 similarity index 100% rename from config/keys/publickey0 rename to config/keysRSA/publickey0 diff --git a/config/keys/publickey1 b/config/keysRSA/publickey1 similarity index 100% rename from config/keys/publickey1 rename to config/keysRSA/publickey1 diff --git a/config/keys/publickey1001 b/config/keysRSA/publickey1001 similarity index 100% rename from config/keys/publickey1001 rename to config/keysRSA/publickey1001 diff --git a/config/keys/publickey2 b/config/keysRSA/publickey2 similarity index 100% rename from config/keys/publickey2 rename to config/keysRSA/publickey2 diff --git a/config/keys/publickey3 b/config/keysRSA/publickey3 similarity index 100% rename from config/keys/publickey3 rename to config/keysRSA/publickey3 diff --git a/config/keysSSL_TLS/EC_KeyPair_256.pkcs12 b/config/keysSSL_TLS/EC_KeyPair_256.pkcs12 new file mode 100644 index 000000000..008dbd8be Binary files /dev/null and b/config/keysSSL_TLS/EC_KeyPair_256.pkcs12 differ diff --git a/config/keysSSL_TLS/EC_KeyPair_384.pkcs12 b/config/keysSSL_TLS/EC_KeyPair_384.pkcs12 new file mode 100644 index 000000000..d9c10e000 Binary files /dev/null and b/config/keysSSL_TLS/EC_KeyPair_384.pkcs12 differ diff --git a/config/keysSSL_TLS/RSA_KeyPair_1024.pkcs12 b/config/keysSSL_TLS/RSA_KeyPair_1024.pkcs12 new file mode 100644 index 000000000..dd03122b7 Binary files /dev/null and b/config/keysSSL_TLS/RSA_KeyPair_1024.pkcs12 differ diff --git a/config/keysSSL_TLS/RSA_KeyPair_2048.pkcs12 b/config/keysSSL_TLS/RSA_KeyPair_2048.pkcs12 new file mode 100644 index 000000000..d41fdbbea Binary files /dev/null and b/config/keysSSL_TLS/RSA_KeyPair_2048.pkcs12 differ diff --git a/config/keysSunEC/privatekey0 b/config/keysSunEC/privatekey0 new file mode 100644 index 000000000..a286a5450 --- /dev/null +++ b/config/keysSunEC/privatekey0 @@ -0,0 +1 @@ +MEECAQAwEwYHKoZIzj0CAQYIKoZIzj0DAQcEJzAlAgEBBCCWnmor2QfcYpvkoxfP/JlWhSfC9J/yGSRWHHiZx6sL9Q== \ No newline at end of file diff --git a/config/keysSunEC/privatekey1 b/config/keysSunEC/privatekey1 new file mode 100644 index 000000000..d4dc1be19 --- /dev/null +++ b/config/keysSunEC/privatekey1 @@ -0,0 +1 @@ +MEECAQAwEwYHKoZIzj0CAQYIKoZIzj0DAQcEJzAlAgEBBCCbY/slFi0bqgns3p7XMfl5Xn5lLnX5ViwICGV1iK8zUQ== \ No newline at end of file diff --git a/config/keysSunEC/privatekey1001 b/config/keysSunEC/privatekey1001 new file mode 100644 index 000000000..c1fa495a9 --- /dev/null +++ b/config/keysSunEC/privatekey1001 @@ -0,0 +1 @@ +MEECAQAwEwYHKoZIzj0CAQYIKoZIzj0DAQcEJzAlAgEBBCBIiauyq/F7n1hOqomza0bl1r+sp0Owpb/O+JnmpD3O1g== \ No newline at end of file diff --git a/config/keysSunEC/privatekey2 b/config/keysSunEC/privatekey2 new file mode 100644 index 000000000..43d553b15 --- /dev/null +++ b/config/keysSunEC/privatekey2 @@ -0,0 +1 @@ +MEECAQAwEwYHKoZIzj0CAQYIKoZIzj0DAQcEJzAlAgEBBCBXn3kUjdMlGL+Ayrt7PUDr97p+4FgpmZ6M69qAOcwlWA== \ No newline at end of file diff --git a/config/keysSunEC/privatekey3 b/config/keysSunEC/privatekey3 new file mode 100644 index 000000000..50f0ec4a7 --- /dev/null +++ b/config/keysSunEC/privatekey3 @@ -0,0 +1 @@ +MEECAQAwEwYHKoZIzj0CAQYIKoZIzj0DAQcEJzAlAgEBBCAtJ6SnBswCvaK6HcC2T7R1Z/NQdRSYVzKRSWeY/LVShA== \ No newline at end of file diff --git a/config/keysSunEC/publickey0 b/config/keysSunEC/publickey0 new file mode 100644 index 000000000..e3ef32b1b --- /dev/null +++ b/config/keysSunEC/publickey0 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEnj/UYaihskrz75tjovBB6a4JJMmxN1u5WNFDVhIH2f6WHVuAzVHnBw1nmWnFEh8qUK6Jarlekfw7HML1l+N9Rw== \ No newline at end of file diff --git a/config/keysSunEC/publickey1 b/config/keysSunEC/publickey1 new file mode 100644 index 000000000..308d3b33e --- /dev/null +++ b/config/keysSunEC/publickey1 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEpQkXMt5fgOufqui1NbhMj28P2sNOQT15aEUur4TkhzGaHYHl1eYf2iuS2sY1zVNu47ILjpHOOaZmzLFUn/3GkQ== \ No newline at end of file diff --git a/config/keysSunEC/publickey1001 b/config/keysSunEC/publickey1001 new file mode 100644 index 000000000..d58dd960b --- /dev/null +++ b/config/keysSunEC/publickey1001 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE8ArsjzE5ETl/5YSkSCJ0eNONzZjmMxk2dNYPrJRFG5fON6nqp1aHOEJOxrQPtEiVQ8KSGtAWnTHf0891GYpQ7A== \ No newline at end of file diff --git a/config/keysSunEC/publickey2 b/config/keysSunEC/publickey2 new file mode 100644 index 000000000..2779f5325 --- /dev/null +++ b/config/keysSunEC/publickey2 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE8AJL+YVw44YI+mWRLhoIlWlMysZiM34CKaieMqYhgyvNoSbTGMHVk17BzntDQj5wYekCI+QJgHKV1LZ4xX1Wbw== \ No newline at end of file diff --git a/config/keysSunEC/publickey3 b/config/keysSunEC/publickey3 new file mode 100644 index 000000000..7d323c39a --- /dev/null +++ b/config/keysSunEC/publickey3 @@ -0,0 +1 @@ +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAERNFRmBVFZ9g0I6Tk6KsGrdOxr6+e+p0c+jtasHwiK3qt4K6tghI75tHna+0VA34IkDmLgD8p59huf66TtFWVZw== \ No newline at end of file diff --git a/config/logback.xml b/config/logback.xml index 1ed33b439..aecd0df70 100644 --- a/config/logback.xml +++ b/config/logback.xml @@ -1,47 +1,50 @@ - - + %d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n - - + -- %msg%n - + 1000 - - - - - - - - - - - - + - + + + + + + + + + + + + diff --git a/config/system.config b/config/system.config index 88c448027..444c6bf2b 100644 --- a/config/system.config +++ b/config/system.config @@ -16,18 +16,36 @@ ####### Communication Configurations ####### ############################################ -#HMAC algorithm used to authenticate messages between processes ('HmacSHA512' is the default value) -system.communication.hmacAlgorithm = HmacSHA512 +#Algorithm to generate secret keys used to generate MACs ('PBKDF2WithHmacSHA1' is the default value) +system.communication.secretKeyAlgorithm = PBKDF2WithHmacSHA1 +#Secret keys algorithm provider ('SunJCE' is the default value) +system.communication.secretKeyAlgorithmProvider = SunJCE -#Algorithm to generate secret keys used to generate MACs ('PBEWithSHA1AndDES' is the default value) -system.communication.secretKeyAlgorithm = PBEWithSHA1AndDES +#Algorithm used to compute hashes ('SHA-512' is the default value) (MessageDigest) +system.communication.hashAlgorithm = SHA-512 +#Hash algorithm provider ('SUN' is the default value) +system.communication.hashAlgorithmProvider = SUN +# -------- RSA #Signature algorithm used to verify clients requests and to perform the authenticated Diffie-Hellman exchange during -#replica start-up ('SHA512withRSA' is the default value). This parameter is overriden in the event that a custom key loader is supplied -system.communication.signatureAlgorithm = SHA512withRSA +#replica start-up ('SHA512withRSA' is the default value). This parameter is overwritten in the event that a custom key loader is supplied +#https://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#SunRsaSignProvider +#system.communication.signatureAlgorithm = SHA512withRSA +#system.communication.signatureAlgorithmProvider = SunRsaSign +#system.communication.defaultKeyLoader = RSA + +# -------- Elliptic Curve (SunEC) +#Signature algorithm provider ('Elliptic Curve is used with keyLoader ECDSA or SunEC', BC (Bounce Castle) -> ECDSA and SunEC -> SunEC) +#https://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#SunEC +#system.communication.signatureAlgorithm = SHA512withECDSA +#system.communication.signatureAlgorithmProvider = SunEC +#ystem.communication.defaultKeyLoader = SunEC + +# -------- Elliptic Curve (Bouncy Castle) +system.communication.signatureAlgorithm = SHA512withECDSA +system.communication.signatureAlgorithmProvider = BC +system.communication.defaultKeyLoader = ECDSA -# Algorithm used to compute hashes ('SHA-512' is the default value) -system.communication.hashAlgorithm = SHA-512 #Specify if the communication system should use a thread to send data (true or false) system.communication.useSenderThread = true @@ -38,7 +56,7 @@ system.communication.defaultkeys = true #IP address this replica should bind to. If this parameter does not have a valid ip address, #the replica will fetch the host address on its own. If config/hosts.config specifies the -#loopback address for the host machine, this parameter is overriden by that +#loopback address for the host machine, this parameter is overwritten by that system.communication.bindaddress = auto ############################################ @@ -49,21 +67,24 @@ system.communication.bindaddress = auto system.servers.num = 4 #Maximum number of faulty replicas -system.servers.f = 1 +system.servers.f = 1 -#Timeout to asking for a client request -system.totalordermulticast.timeout = 2000 +#Timeout to asking for a client request. This value should be greater than the batch timeout +system.totalordermulticast.timeout = 20000 +#Batch timeout. If set to any non-positive integer value, the next consensus instance +#is triggered as soon as (1) the previous one is finished, and (2) any number new requests arrive, +#without waiting to accumulate more requests for the batch. This value should be lower than the request timeout +system.totalordermulticast.batchtimeout = -1 #Maximum batch size (in number of messages) -system.totalordermulticast.maxbatchsize = 400 +system.totalordermulticast.maxbatchsize = 1024 #Number of nonces (for non-determinism actions) generated system.totalordermulticast.nonces = 10 #if verification of leader-generated timestamps are increasing -#it can only be used on systems in which the network clocks -#are synchronized +#it can only be used on systems in which the network clocks are synchronized system.totalordermulticast.verifyTimestamps = false #Quantity of messages that can be stored in the receive queue of the communication system @@ -72,11 +93,8 @@ system.communication.inQueueSize = 500000 # Quantity of messages that can be stored in the send queue of each replica system.communication.outQueueSize = 500000 -#Set to 1 if SMaRt should use signatures, set to 0 if otherwise -system.communication.useSignatures = 0 - -#Set to 1 if SMaRt should use MAC's, set to 0 if otherwise -system.communication.useMACs = 1 +#Set to true if SMaRt should use signatures, otherwise set to false (default is false). +system.communication.useSignatures = false #Print information about the replica when it is shutdown system.shutdownhook = true @@ -85,6 +103,10 @@ system.shutdownhook = true #This is not the same batch used during the ordering protocol system.samebatchsize = false +#Size of the thread pool that transmits replies to clients. +#If set to 0, no thread pool is used and this done sequentially by the delivery thread instead. +system.numrepliers = 16 + ############################################ ###### State Transfer Configurations ####### ############################################ @@ -111,7 +133,7 @@ system.totalordermulticast.log_to_disk = false system.totalordermulticast.sync_log = false #Period at which BFT-SMaRt requests the state to the application (for the state transfer state protocol) -system.totalordermulticast.checkpoint_period = 1000 +system.totalordermulticast.checkpoint_period = 1024 system.totalordermulticast.global_checkpoint_period = 120000 system.totalordermulticast.checkpoint_to_disk = false @@ -132,3 +154,54 @@ system.ttp.id = 7002 #This sets if the system will function in Byzantine or crash-only mode. Set to "true" to support Byzantine faults system.bft = true + +############################################ +######## SSL/TLS Configurations ########## +######## Author: Tulio A. Ribeiro ########## +############################################ + +#Enable or Disable SSL/TLS, default is true. If SSL/TLS is enabled, it will automatically disable 'useMAC' variable. +#system.ssltls = true # Always true. + +#Note that the standard TLS protocol version names used in the JDK(1.8) are: +#Possible options: 'SSLv3', 'TLSv1', 'TLSv1.1' and 'TLSv1.2'. +#Prefer TLSv1.1 or TLSv1.2 instead the others. (Default is TLSv1.2.). +system.ssltls.protocol_version = TLSv1.2 + +#The command below generates the secret key file. It shall be stored into config/keysSSL_TLS/* +#If you need another password, it is necessary to change the variable SECRET at classes: +#'ServerConnectionSSLTLS' and 'ServersCommunicationLayerSSLTLS' and generate the file with the new password. +## * ##Elliptic Curve +## * $keytool -genkey -keyalg EC -keysize 256 -alias bftsmartEC -keypass MySeCreT_2hMOygBwY -keystore ./ecKeyPair_256.pkcs12 -dname "CN=BFT-SMaRT" +## * $keytool -importkeystore -srckeystore ./ecKeyPair_256.pkcs12 -destkeystore ./ecKeyPair_256.pkcs12 -deststoretype pkcs12 +## * +## * ##RSA +## * $keytool -genkey -keyalg RSA -keysize 2048 -alias bftsmartRSA -keypass MySeCreT_2hMOygBwY -keystore ./RSA_KeyPair_2048.pkcs12 -dname "CN=BFT-SMaRT" +## * $keytool -importkeystore -srckeystore ./RSA_KeyPair_2048.pkcs12 -destkeystore ./RSA_KeyPair_2048.pkcs12 -deststoretype pkcs12 + +#Cipher Suite list enabled by the server. NULL means, NoCipher. +#ECDH(E) - Elliptic Curve Diffie-Hellman (Ephemeral) or Diffie-Hellman Ephemeral (DHE) it is the standard +#recommendation, in general, the recommendation is to use "Ephemeral". See https://ciphersuite.info/cs/ +#The ciphers shall be separately by ",", if there is just one, put a "," at the end (without spaces). + +## ECDSA +system.ssltls.key_store_file=EC_KeyPair_256.pkcs12 +#system.ssltls.key_store_file=EC_KeyPair_384.pkcs12 +system.ssltls.enabled_ciphers = TLS_ECDHE_ECDSA_WITH_NULL_SHA, +# With cipher (recommended). +#system.ssltls.enabled_ciphers = TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, + +## NOTE: it is mandatory to use the specific keyPair file regarding the algorithm, see below. +## TLS_ECDHE_ECDSA_WITH_*, // used with EC keyStore (EC_KeyPair_*.pkcs12). +## TLS_RSA_WITH_*, OR TLS_ECDHE_RSA_WITH_*, // used with RSA keyStore (RSA_KeyPair_*.pkcs12). + +## RSA +#system.ssltls.key_store_file=RSA_KeyPair_1024.pkcs12 +#system.ssltls.key_store_file=RSA_KeyPair_2048.pkcs12 +#system.ssltls.enabled_ciphers = TLS_RSA_WITH_NULL_SHA256, +# With cipher (recommended) +##system.ssltls.enabled_ciphers = TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, +#With cipher (secure) +##system.ssltls.enabled_ciphers = TLS_RSA_WITH_AES_128_GCM_SHA256, + + diff --git a/doc/allclasses-frame.html b/doc/allclasses-frame.html index 87ea0965a..8128ac6ed 100644 --- a/doc/allclasses-frame.html +++ b/doc/allclasses-frame.html @@ -2,9 +2,9 @@ - + All Classes - + @@ -19,18 +19,17 @@

All Classes

  • DurabilityCoordinator
  • Executable
  • Extractor
    • -
  • FIFOExecutable
    • +
  • KeyLoader
  • MessageContext
  • Recoverable
  • ReplicaContext
  • Replier
  • RequestContext
  • RequestVerifier
    • -
  • RSAKeyPairGenerator
  • ServiceProxy
  • ServiceReplica
  • SingleExecutable
    • -
  • TOMSender
    • +
  • VMServices
    • diff --git a/doc/allclasses-noframe.html b/doc/allclasses-noframe.html index 8ead85fe3..f515a05f2 100644 --- a/doc/allclasses-noframe.html +++ b/doc/allclasses-noframe.html @@ -2,9 +2,9 @@ - + All Classes - + @@ -19,18 +19,17 @@

    All Classes

  • DurabilityCoordinator
  • Executable
  • Extractor
    • -
  • FIFOExecutable
    • +
  • KeyLoader
  • MessageContext
  • Recoverable
  • ReplicaContext
  • Replier
  • RequestContext
  • RequestVerifier
    • -
  • RSAKeyPairGenerator
  • ServiceProxy
  • ServiceReplica
  • SingleExecutable
    • -
  • TOMSender
    • +
  • VMServices
    • diff --git a/doc/bftsmart/tom/server/defaultservices/DefaultReplier.html b/doc/bftsmart/reconfiguration/VMServices.html similarity index 57% rename from doc/bftsmart/tom/server/defaultservices/DefaultReplier.html rename to doc/bftsmart/reconfiguration/VMServices.html index 95cac7c5a..083740291 100644 --- a/doc/bftsmart/tom/server/defaultservices/DefaultReplier.html +++ b/doc/bftsmart/reconfiguration/VMServices.html @@ -2,17 +2,17 @@ - -DefaultReplier - - - + +VMServices + + + + + +

    bftsmart.reconfiguration

    +
    +

    Classes

    + +
    + + diff --git a/doc/bftsmart/reconfiguration/package-summary.html b/doc/bftsmart/reconfiguration/package-summary.html new file mode 100644 index 000000000..2a533dc6e --- /dev/null +++ b/doc/bftsmart/reconfiguration/package-summary.html @@ -0,0 +1,142 @@ + + + + + +bftsmart.reconfiguration + + + + + + + + +
    + + +
    Skip navigation links
    + + + + +
    +
    + + + +
    + +
    + + +
    + +
    +

    Package bftsmart.reconfiguration

    +
    +
    +
      +
    • + + + + + + + + + + + + +
      Class Summary 
      ClassDescription
      VMServices +
      This class is used by the trusted client to add and remove replicas from the group
      +
      +
      • +
    +
    + +
    + + +
    Skip navigation links
    + + + + +
    +
    + + + +
    + +
    + + +
    + + + diff --git a/doc/bftsmart/reconfiguration/package-tree.html b/doc/bftsmart/reconfiguration/package-tree.html new file mode 100644 index 000000000..ea9e4876d --- /dev/null +++ b/doc/bftsmart/reconfiguration/package-tree.html @@ -0,0 +1,135 @@ + + + + + +bftsmart.reconfiguration Class Hierarchy + + + + + + + + +
    + + +
    Skip navigation links
    + + + + +
    +
    + + + +
    + +
    + + +
    + +
    +

    Hierarchy For Package bftsmart.reconfiguration

    +Package Hierarchies: + +
    +
    +

    Class Hierarchy

    +
      +
    • java.lang.Object + +
      • +
    +
    + +
    + + +
    Skip navigation links
    + + + + +
    +
    + + + +
    + +
    + + +
    + + + diff --git a/doc/bftsmart/tom/AsynchServiceProxy.html b/doc/bftsmart/tom/AsynchServiceProxy.html index bdca587b7..01ebe1256 100644 --- a/doc/bftsmart/tom/AsynchServiceProxy.html +++ b/doc/bftsmart/tom/AsynchServiceProxy.html @@ -2,9 +2,9 @@ - + AsynchServiceProxy - + @@ -99,7 +99,7 @@

    Class AsynchServiceProxy

  • java.lang.Object
  • - + + + +
      +
    • +

      AsynchServiceProxy

      +
      public AsynchServiceProxy(int processId,
+                          java.lang.String configHome,
+                          KeyLoader loader)
      +
      Constructor
      +
      +
      See Also:
      +
      bellow
      +
      +
      • +
    +
      @@ -282,7 +317,19 @@

      AsynchServiceProxy

      public AsynchServiceProxy(int processId,
                           java.lang.String configHome,
                           java.util.Comparator<byte[]> replyComparator,
-                          Extractor replyExtractor)
      + Extractor replyExtractor, + KeyLoader loader) +
      Constructor
      +
      +
      Parameters:
      +
      processId - Process id for this client (should be different from replicas)
      +
      configHome - Configuration directory for BFT-SMART
      +
      replyComparator - Used for comparing replies from different servers + to extract one returned by f+1
      +
      replyExtractor - Used for extracting the response from the matching + quorum of replies
      +
      loader - Used to load signature keys from disk
      +
    • @@ -303,11 +350,8 @@

    invokeAsynchRequest

    bftsmart.communication.client.ReplyListener replyListener, bftsmart.tom.core.messages.TOMMessageType reqType)
    -
    Parameters:
    -
    request -
    -
    replyListener -
    -
    reqType - Request type
    -
    Returns:
    +
    See Also:
    +
    bellow
    @@ -321,13 +365,15 @@

    invokeAsynchRequest

    int[] targets, bftsmart.communication.client.ReplyListener replyListener, bftsmart.tom.core.messages.TOMMessageType reqType) +
    This method asynchronously sends a request to the replicas.
    Parameters:
    -
    request -
    -
    targets -
    -
    replyListener -
    +
    request - Request to be sent
    +
    targets - The IDs for the replicas to which to send the request
    +
    replyListener - Callback object that handles reception of replies
    reqType - Request type
    Returns:
    +
    A unique identification for the request
    @@ -338,9 +384,11 @@

    invokeAsynchRequest

  • cleanAsynchRequest

    public void cleanAsynchRequest(int requestId)
    +
    Purges all information associated to the request. + This should always be invoked once enough replies are received and processed by the ReplyListener callback.
    Parameters:
    -
    requestId - Request
    +
    requestId - A unique identification for a previously sent request
    • @@ -351,7 +399,6 @@

    cleanAsynchRequest

  • replyReceived

    public void replyReceived(bftsmart.tom.core.messages.TOMMessage reply)
    -
    Description copied from class: ServiceProxy
    This is the method invoked by the client side communication system.
    Specified by:
    diff --git a/doc/bftsmart/tom/MessageContext.html b/doc/bftsmart/tom/MessageContext.html index 7a4c1ab5b..816b4a979 100644 --- a/doc/bftsmart/tom/MessageContext.html +++ b/doc/bftsmart/tom/MessageContext.html @@ -2,9 +2,9 @@ - + MessageContext - + @@ -18,8 +18,8 @@ catch(err) { } //--> -var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10,"i6":10,"i7":10,"i8":10,"i9":10,"i10":10,"i11":10,"i12":9,"i13":10,"i14":10,"i15":10,"i16":10,"i17":10,"i18":10,"i19":10,"i20":10,"i21":10}; -var tabs = {65535:["t0","All Methods"],1:["t1","Static Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10,"i6":10,"i7":10,"i8":10,"i9":10,"i10":10,"i11":10,"i12":10,"i13":10,"i14":10,"i15":10,"i16":10,"i17":10,"i18":10,"i19":10,"i20":42}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"],32:["t6","Deprecated Methods"]}; var altColor = "altColor"; var rowColor = "rowColor"; var tableTab = "tableTab"; @@ -189,102 +189,138 @@

    Constructor Summary

    Method Summary

    - + - + - + - + - + - + - + - + - + - + - + - + - - + + - - + + - - + + - - + + - - + + - - + + - + - - - - - + - +
    All Methods Static Methods Instance Methods Concrete Methods All Methods Instance Methods Concrete Methods Deprecated Methods 
    Modifier and Type Method and Description
    intgetConsensusId() getConsensusId() +
    Returns the ID of the consensus in which this request was ordered (-1 + if readonly).
    +
    bftsmart.tom.core.messages.TOMMessagegetFirstInBatch() getFirstInBatch() +
    Returns the first request in the ordered batch
    +
    intgetLeader() getLeader() +
    Returns the id of the leader replica in which this message was ordered + (-1 if readonly).
    +
    byte[]getNonces() getNonces() +
    Returns the nonces as generated by the leader
    +
    intgetNumOfNonces() getNumOfNonces() +
    Returns the number of nonces as generated by the leader
    +
    intgetOperationId() getOperationId() +
    Returns the operation ID
    +
    java.util.Set<bftsmart.consensus.messages.ConsensusMessage>getProof() getProof() +
    Returns the proof for the consensus.
    +
    intgetRegency() getRegency() +
    Returns the current regency in which the message was ordered.
    +
    intgetReplyServer() getReplyServer() +
    Returns the ID of the server expected to send a non-hashed reply when the clients sends an unordered hashed request.
    +
    longgetSeed() getSeed() +
    Returns the seed as generated by the leader
    +
    int getSender() -
    Returns the sender of the message
    +
    Returns the ID of the client that sent the message
    intgetSequence() getSequence() +
    Returns the request's sequence number
    +
    static longgetSerialVersionUID() intgetSession() +
    Returns the client's session
    +
    intgetSession() byte[]getSignature() +
    Returns the client's signature.
    +
    byte[]getSignature() longgetTimestamp() +
    Returns the timestamp as generated by the leader
    +
    longgetTimestamp() bftsmart.tom.core.messages.TOMMessageTypegetType() +
    Returns the request type
    +
    bftsmart.tom.core.messages.TOMMessageTypegetType() intgetViewID() +
    Returns the view ID
    +
    intgetViewID() booleanisLastInBatch() +
    Returns the last request in the ordered batch
    +
    booleanisLastInBatch() isNoOp() +
    Returns true if the consensus instance has no operations to deliver to the application
    +
    booleanisNoOp() 
    bftsmart.tom.core.messages.TOMMessage recreateTOMMessage(byte[] content)
    Generates a TOMMessage for its associated requests using the new info that it now supports since the previous commit.
    voidsetLastInBatch() setLastInBatch() +
    Deprecated.  
    +
      @@ -351,23 +387,23 @@

      MessageContext

      Constructor
      Parameters:
      -
      sender -
      -
      viewID -
      -
      type -
      -
      session -
      -
      sequence -
      -
      operationId -
      -
      replyServer -
      -
      signature -
      -
      timestamp -
      -
      numOfNonces -
      -
      seed -
      -
      regency -
      -
      leader -
      -
      consensusId -
      -
      proof -
      -
      firstInBatch -
      -
      noOp -
      +
      sender - The sender ID
      +
      viewID - The view ID
      +
      type - The request type
      +
      session - Client's session
      +
      sequence - Request's sequence number
      +
      operationId - The operation ID
      +
      replyServer - ID of the server expected to send a non-hashed reply when the clients sends an unordered hashed request.
      +
      signature - Client's signature.
      +
      timestamp - The timestamp as generated by the leader
      +
      numOfNonces - The nonces as generated by the leader
      +
      seed - The seed as generated by the leader
      +
      regency - The current regency in which the message was ordered.
      +
      leader - The leader with which the batch was decided
      +
      consensusId - The ID of the consensus in which this request was ordered
      +
      proof - The proof for the consensus
      +
      firstInBatch - The first request in the ordered batch
      +
      noOp - true if the consensus instance has no operations to deliver to the application
    @@ -379,15 +415,6 @@

    MessageContext

    Method Detail

    - - - -
      -
    • -

      getSerialVersionUID

      -
      public static long getSerialVersionUID()
      -
      • -
    @@ -395,6 +422,11 @@

    getSerialVersionUID

  • getViewID

    public int getViewID()
    +
    Returns the view ID
    +
    +
    Returns:
    +
    The view ID
    +
    • @@ -404,6 +436,11 @@

    getViewID

  • getType

    public bftsmart.tom.core.messages.TOMMessageType getType()
    +
    Returns the request type
    +
    +
    Returns:
    +
    The request type
    +
    •  @@ -413,6 +450,11 @@

    getType

  • getSession

    public int getSession()
    +
    Returns the client's session
    +
    +
    Returns:
    +
    Client's session
    +
    •  @@ -422,6 +464,11 @@

    getSession

  • getSequence

    public int getSequence()
    +
    Returns the request's sequence number
    +
    +
    Returns:
    +
    Request's sequence number
    +
    •  @@ -431,6 +478,11 @@

    getSequence

  • getOperationId

    public int getOperationId()
    +
    Returns the operation ID
    +
    +
    Returns:
    +
    The operation ID
    +
    •  @@ -440,6 +492,11 @@

    getOperationId

  • getReplyServer

    public int getReplyServer()
    +
    Returns the ID of the server expected to send a non-hashed reply when the clients sends an unordered hashed request.
    +
    +
    Returns:
    +
    ID of the server expected to send a non-hashed reply when the clients sends an unordered hashed request.
    +
    •  @@ -449,6 +506,11 @@

    getReplyServer

  • getSignature

    public byte[] getSignature()
    +
    Returns the client's signature.
    +
    +
    Returns:
    +
    Client's signature.
    +
    •  @@ -458,10 +520,10 @@

    getSignature

  • getSender

    public int getSender()
    -
    Returns the sender of the message
    +
    Returns the ID of the client that sent the message
    Returns:
    -
    The sender of the message
    +
    The sender ID
    • @@ -472,9 +534,10 @@

    getSender

  • getTimestamp

    public long getTimestamp()
    +
    Returns the timestamp as generated by the leader
    Returns:
    -
    the timestamp
    +
    The timestamp as generated by the leader
    • @@ -485,9 +548,10 @@

    getTimestamp

  • getNonces

    public byte[] getNonces()
    +
    Returns the nonces as generated by the leader
    Returns:
    -
    the nonces
    +
    The nonces as generated by the leader
    • @@ -498,6 +562,11 @@

    getNonces

  • getNumOfNonces

    public int getNumOfNonces()
    +
    Returns the number of nonces as generated by the leader
    +
    +
    Returns:
    +
    the number of nonces
    +
    •  @@ -507,6 +576,11 @@

    getNumOfNonces

  • getSeed

    public long getSeed()
    +
    Returns the seed as generated by the leader
    +
    +
    Returns:
    +
    The seed as generated by the leader
    +
    •  @@ -516,9 +590,11 @@

    getSeed

  • getConsensusId

    public int getConsensusId()
    +
    Returns the ID of the consensus in which this request was ordered (-1 + if readonly).
    Returns:
    -
    the consensusId
    +
    The ID of the consensus in which this request was ordered
    • @@ -529,9 +605,11 @@

    getConsensusId

  • getLeader

    public int getLeader()
    +
    Returns the id of the leader replica in which this message was ordered + (-1 if readonly).
    Returns:
    -
    the leader with which the batch was decided
    +
    The leader with which the batch was decided
    • @@ -542,9 +620,10 @@

    getLeader

  • getProof

    public java.util.Set<bftsmart.consensus.messages.ConsensusMessage> getProof()
    +
    Returns the proof for the consensus.
    Returns:
    -
    the proof for the consensus
    +
    The proof for the consensus
    • @@ -555,9 +634,10 @@

    getProof

  • getRegency

    public int getRegency()
    +
    Returns the current regency in which the message was ordered.
    Returns:
    -
    the regency
    +
    The current regency in which the message was ordered.
    • @@ -568,9 +648,10 @@

    getRegency

  • getFirstInBatch

    public bftsmart.tom.core.messages.TOMMessage getFirstInBatch()
    +
    Returns the first request in the ordered batch
    Returns:
    -
    the first message in the ordered batch
    +
    The first request in the ordered batch
    • @@ -581,6 +662,7 @@

    getFirstInBatch

  • setLastInBatch

    public void setLastInBatch()
    +
    Deprecated. 
    •  @@ -590,6 +672,11 @@

    setLastInBatch

  • isLastInBatch

    public boolean isLastInBatch()
    +
    Returns the last request in the ordered batch
    +
    +
    Returns:
    +
    The last request in the ordered batch
    +
    •  @@ -599,6 +686,11 @@

    isLastInBatch

  • isNoOp

    public boolean isNoOp()
    +
    Returns true if the consensus instance has no operations to deliver to the application
    +
    +
    Returns:
    +
    true if the consensus instance has no operations to deliver to the application
    +
    •  diff --git a/doc/bftsmart/tom/ReplicaContext.html b/doc/bftsmart/tom/ReplicaContext.html index 647ad834c..463d7f2bd 100644 --- a/doc/bftsmart/tom/ReplicaContext.html +++ b/doc/bftsmart/tom/ReplicaContext.html @@ -2,9 +2,9 @@ - + ReplicaContext - + @@ -18,7 +18,7 @@ catch(err) { } //--> -var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10}; +var methods = {"i0":10,"i1":10,"i2":10,"i3":10}; var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; var altColor = "altColor"; var rowColor = "rowColor"; @@ -130,7 +130,9 @@

    Constructor Summary

    ReplicaContext(bftsmart.communication.ServerCommunicationSystem cs, - bftsmart.reconfiguration.ServerViewController SVController)  + bftsmart.reconfiguration.ServerViewController SVController) +
    Constructor
    +
    • @@ -155,7 +157,9 @@

    Method Summary

    bftsmart.communication.ServerCommunicationSystem -getServerCommunicationSystem()  +getServerCommunicationSystem() +
    Returns the replica's communication system
    + bftsmart.reconfiguration.util.TOMConfiguration @@ -169,10 +173,6 @@

    Method Summary

    Returns the controller of the replica's view
    - -void -setServerCommunicationSystem(bftsmart.communication.ServerCommunicationSystem cs)  - @@ -258,19 +264,15 @@

    getCurrentView

     -
      +

      • getServerCommunicationSystem

        public bftsmart.communication.ServerCommunicationSystem getServerCommunicationSystem()
        -
        • -
      - - - -
        -
      • -

        setServerCommunicationSystem

        -
        public void setServerCommunicationSystem(bftsmart.communication.ServerCommunicationSystem cs)
        +
        Returns the replica's communication system
        +
        +
        Returns:
        +
        The replica's communication system
        +
      diff --git a/doc/bftsmart/tom/RequestContext.html b/doc/bftsmart/tom/RequestContext.html index 88be837a8..a21d34c78 100644 --- a/doc/bftsmart/tom/RequestContext.html +++ b/doc/bftsmart/tom/RequestContext.html @@ -2,9 +2,9 @@ - + RequestContext - + @@ -18,7 +18,7 @@ catch(err) { } //--> -var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10}; +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10,"i6":10}; var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; var altColor = "altColor"; var rowColor = "rowColor"; @@ -129,12 +129,15 @@

      Constructor Summary

      Constructor and Description -RequestContext(int reqId, +RequestContext(int reqId, int operationId, bftsmart.tom.core.messages.TOMMessageType requestType, int[] targets, long sendingTime, - bftsmart.communication.client.ReplyListener replyListener)  + bftsmart.communication.client.ReplyListener replyListener, + byte[] request) +
      Constructor
      + @@ -153,27 +156,45 @@

      Method Summary

      int -getOperationId()  +getOperationId() +
      Returns the operation ID
      + bftsmart.communication.client.ReplyListener -getReplyListener()  +getReplyListener() +
      Returns the reply listener associated to the request
      + int -getReqId()  +getReqId() +
      Returns the request unique ID
      + -bftsmart.tom.core.messages.TOMMessageType -getRequestType()  +byte[] +getRequest() +
      The payload of the request
      + -long -getSendingTime()  +bftsmart.tom.core.messages.TOMMessageType +getRequestType() +
      Returns the request type
      + +long +getSendingTime() +
      Returns the sending time
      + + + int[] -getTargets()  +getTargets() +
      Returns the IDs of the targets to which the request was sent
      +
        @@ -197,7 +218,7 @@

        Methods inherited from class java.lang.Object

        Constructor Detail

        - +
          @@ -208,7 +229,19 @@

          RequestContext

          bftsmart.tom.core.messages.TOMMessageType requestType, int[] targets, long sendingTime, - bftsmart.communication.client.ReplyListener replyListener) + bftsmart.communication.client.ReplyListener replyListener, + byte[] request) +
          Constructor
          +
          +
          Parameters:
          +
          reqId - Request ID
          +
          operationId - Operation ID
          +
          requestType - The request type
          +
          targets - IDs of the targets to which the request was sent
          +
          sendingTime - Sending time
          +
          replyListener - Reply listener associated to the request
          +
          request - Payload of the request
          +
        @@ -226,6 +259,11 @@

        Method Detail

      • getReqId

        public final int getReqId()
        +
        Returns the request unique ID
        +
        +
        Returns:
        +
        Request ID
        +
      @@ -235,6 +273,11 @@

      getReqId

    • getOperationId

      public final int getOperationId()
      +
      Returns the operation ID
      +
      +
      Returns:
      +
      Operation ID
      +
    @@ -244,6 +287,11 @@

    getOperationId

  • getRequestType

    public final bftsmart.tom.core.messages.TOMMessageType getRequestType()
    +
    Returns the request type
    +
    +
    Returns:
    +
    The request type
    +
    •  @@ -253,6 +301,11 @@

    getRequestType

  • getSendingTime

    public final long getSendingTime()
    +
    Returns the sending time
    +
    +
    Returns:
    +
    Sending time
    +
    •  @@ -262,15 +315,39 @@

    getSendingTime

  • getReplyListener

    public bftsmart.communication.client.ReplyListener getReplyListener()
    +
    Returns the reply listener associated to the request
    +
    +
    Returns:
    +
    Reply listener associated to the request
    +
    •  -
      +

      • getTargets

        public int[] getTargets()
        +
        Returns the IDs of the targets to which the request was sent
        +
        +
        Returns:
        +
        IDs of the targets to which the request was sent
        +
        +
        • +
      + + + +
        +
      • +

        getRequest

        +
        public byte[] getRequest()
        +
        The payload of the request
        +
        +
        Returns:
        +
        Payload of the request
        +
      diff --git a/doc/bftsmart/tom/ServiceProxy.html b/doc/bftsmart/tom/ServiceProxy.html index bef13776f..b97415b99 100644 --- a/doc/bftsmart/tom/ServiceProxy.html +++ b/doc/bftsmart/tom/ServiceProxy.html @@ -2,9 +2,9 @@ - + ServiceProxy - + @@ -18,8 +18,8 @@ catch(err) { } //--> -var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10,"i6":10,"i7":10,"i8":10}; -var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10,"i6":10,"i7":42,"i8":10,"i9":10,"i10":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"],32:["t6","Deprecated Methods"]}; var altColor = "altColor"; var rowColor = "rowColor"; var tableTab = "tableTab"; @@ -99,7 +99,7 @@

      Class ServiceProxy

    • java.lang.Object
      • -
      • bftsmart.tom.TOMSender
        • +
      • bftsmart.tom.core.TOMSender
      • - + + + +
          +
        • +

          ServiceProxy

          +
          public ServiceProxy(int processId,
+                    java.lang.String configHome,
+                    KeyLoader loader)
          +
          Constructor
          +
          +
          See Also:
          +
          bellow
          +
          +
          • +
        +
          @@ -341,16 +389,18 @@

          ServiceProxy

          public ServiceProxy(int processId,
                     java.lang.String configHome,
                     java.util.Comparator<byte[]> replyComparator,
-                    Extractor replyExtractor)
          + Extractor replyExtractor, + KeyLoader loader)
          Constructor
          Parameters:
          processId - Process id for this client (should be different from replicas)
          configHome - Configuration directory for BFT-SMART
          -
          replyComparator - used for comparing replies from different servers +
          replyComparator - Used for comparing replies from different servers to extract one returned by f+1
          -
          replyExtractor - used for extracting the response from the matching +
          replyExtractor - Used for extracting the response from the matching quorum of replies
          +
          loader - Used to load signature keys from disk
        @@ -373,7 +423,7 @@

        getInvokeTimeout

        servers replies before returning null.
        Returns:
        -
        the invokeTimeout
        +
        the timeout value in seconds
      @@ -384,6 +434,12 @@

      getInvokeTimeout

    • getInvokeUnorderedHashedTimeout

      public int getInvokeUnorderedHashedTimeout()
      +
      Get the amount of time (in seconds) that this proxy will wait for + servers unordered hashed replies before returning null.
      +
      +
      Returns:
      +
      the timeout value in seconds
      +
    @@ -397,7 +453,7 @@

    setInvokeTimeout

    servers replies before returning null.
    Parameters:
    -
    invokeTimeout - the invokeTimeout to set
    +
    invokeTimeout - the timeout value to set
    @@ -408,6 +464,12 @@

    setInvokeTimeout

  • setInvokeUnorderedHashedTimeout

    public void setInvokeUnorderedHashedTimeout(int timeout)
    +
    Set the amount of time (in seconds) that this proxy will wait for + servers unordered hashed replies before returning null.
    +
    +
    Parameters:
    +
    timeout - the timeout value to set
    +
    •  @@ -417,6 +479,15 @@

    setInvokeUnorderedHashedTimeout

  • invokeOrdered

    public byte[] invokeOrdered(byte[] request)
    +
    This method sends an ordered request to the replicas, and returns the related reply. + If the servers take more than invokeTimeout seconds the method returns null. + This method is thread-safe.
    +
    +
    Parameters:
    +
    request - to be sent
    +
    Returns:
    +
    The reply from the replicas related to request
    +
    •  @@ -426,6 +497,15 @@

    invokeOrdered

  • invokeUnordered

    public byte[] invokeUnordered(byte[] request)
    +
    This method sends an unordered request to the replicas, and returns the related reply. + If the servers take more than invokeTimeout seconds the method returns null. + This method is thread-safe.
    +
    +
    Parameters:
    +
    request - to be sent
    +
    Returns:
    +
    The reply from the replicas related to request
    +
    •  @@ -435,6 +515,17 @@

    invokeUnordered

  • invokeUnorderedHashed

    public byte[] invokeUnorderedHashed(byte[] request)
    +
    This method sends an unordered request to the replicas, and returns the related reply. + This method chooses randomly one replica to send the complete response, while the others + only send a hash of that response. + If the servers take more than invokeTimeout seconds the method returns null. + This method is thread-safe.
    +
    +
    Parameters:
    +
    request - to be sent
    +
    Returns:
    +
    The reply from the replicas related to request
    +
    •  @@ -451,17 +542,27 @@

    invoke

    Parameters:
    request - Request to be sent
    -
    reqType - TOM_NORMAL_REQUESTS for service requests, and other for - reconfig requests.
    +
    reqType - ORDERED_REQUEST/UNORDERED_REQUEST/UNORDERED_HASHED_REQUEST for normal requests, and RECONFIG for + reconfiguration requests.
    Returns:
    The reply from the replicas related to request
    +     + + +
      +
    • +

      reconfigureTo

      +
      protected void reconfigureTo(bftsmart.reconfiguration.views.View v)
      +
      Deprecated. 
      +
      • +
    -
      +

      • replyReceived

        public void replyReceived(bftsmart.tom.core.messages.TOMMessage reply)
        @@ -472,6 +573,20 @@

        replyReceived

      + + + +
        +
      • +

        getReplyQuorum

        +
        protected int getReplyQuorum()
        +
        Retrieves the required quorum size for the amount of replies
        +
        +
        Returns:
        +
        The quorum size for the amount of replies
        +
        +
        • +
    diff --git a/doc/bftsmart/tom/ServiceReplica.html b/doc/bftsmart/tom/ServiceReplica.html index 3a2eecda3..841ba138b 100644 --- a/doc/bftsmart/tom/ServiceReplica.html +++ b/doc/bftsmart/tom/ServiceReplica.html @@ -2,9 +2,9 @@ - + ServiceReplica - + @@ -18,8 +18,8 @@ catch(err) { } //--> -var methods = {"i0":10,"i1":10,"i2":10,"i3":10,"i4":10,"i5":10}; -var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"]}; +var methods = {"i0":10,"i1":10,"i2":10,"i3":42,"i4":10,"i5":42,"i6":42,"i7":10}; +var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],8:["t4","Concrete Methods"],32:["t6","Deprecated Methods"]}; var altColor = "altColor"; var rowColor = "rowColor"; var tableTab = "tableTab"; @@ -49,7 +49,7 @@
    @@ -376,12 +399,30 @@

    joinMsgReceived

    receiveReadonlyMessage

    public final void receiveReadonlyMessage(bftsmart.tom.core.messages.TOMMessage message,
                                          MessageContext msgCtx)
    -
    This message delivers a readonly message, i.e., a message that was not - ordered to the replica and gather the reply to forward to the client
    -
    -
    Parameters:
    -
    message - the request received from the delivery thread
    -
    +
    Deprecated. 
    + + + + + +
      +
    • +

      kill

      +
      public void kill()
      +
      Stops the service execution at a replica. It will shutdown all threads, stop the requests' timer, and drop all enqueued requests, + thus letting the ServiceReplica object be garbage-collected. From the perspective of the rest of the system, this is equivalent + to a simple crash fault.
      +
      • +
    + + + +
      +
    • +

      restart

      +
      public void restart()
      +
      Cleans the object state and reboots execution. From the perspective of the rest of the system, + this is equivalent to a rash followed by a recovery.
    @@ -395,30 +436,49 @@

    receiveMessages

    int[] leaders, bftsmart.tom.leaderchange.CertifiedDecision[] cDecs, bftsmart.tom.core.messages.TOMMessage[][] requests) +
    Deprecated. 
    + + +     + + +
      +
    • +

      getReplicaContext

      +
      public final ReplicaContext getReplicaContext()
      +
      Obtains the current replica context (getting access to several + information and capabilities of the replication engine).
      +
      +
      Returns:
      +
      this replica context
      +
    - +
    • -

      leave

      -
      public void leave()
      -
      This method makes the replica leave the group
      +

      getServerCommunicationSystem

      +
      public bftsmart.communication.ServerCommunicationSystem getServerCommunicationSystem()
      +
      Obtains the current replica communication system.
      +
      +
      Returns:
      +
      The replica's communication system
      +
    - +
    • -

      getReplicaContext

      -
      public final ReplicaContext getReplicaContext()
      -
      Obtains the current replica context (getting access to several - information and capabilities of the replication engine).
      +

      getId

      +
      public int getId()
      +
      Replica ID
      Returns:
      -
      this replica context
      +
      Replica ID
    @@ -450,7 +510,7 @@

    getReplicaContext

    • Frames
      • diff --git a/doc/bftsmart/tom/TOMSender.html b/doc/bftsmart/tom/TOMSender.html deleted file mode 100644 index f5c02f518..000000000 --- a/doc/bftsmart/tom/TOMSender.html +++ /dev/null @@ -1,513 +0,0 @@ - - - - - -TOMSender - - - - - - - - -
      - - -
      Skip navigation links
      - - - - -
      -
      - - - -
      - -
      -
      -
        -
      • Summary: 
        • -
      • Nested | 
        • -
      • Field | 
        • -
      • Constr | 
        • -
      • Method
        • -
      - -
      - - -
      - - -
      -
      bftsmart.tom
      -

      Class TOMSender

      -
      -
      -
        -
      • java.lang.Object
        • -
      • -
          -
        • bftsmart.tom.TOMSender
          • -
        -
        • -
      -
      -
        -
      • -
        -
        All Implemented Interfaces:
        -
        bftsmart.communication.client.ReplyReceiver
        -
        -
        -
        Direct Known Subclasses:
        -
        ServiceProxy
        -
        -
        -
        -
        public abstract class TOMSender
-extends java.lang.Object
-implements bftsmart.communication.client.ReplyReceiver
        -
        This class is used to multicast messages to replicas and receive replies.
        -
        • -
      -
      -
      -
        -
      • - -
          -
        • - - -

          Constructor Summary

          - - - - - - - - -
          Constructors 
          Constructor and Description
          TOMSender() -
          Creates a new instance of TOMulticastSender - - TODO: This may really be empty?
          -
          -
          • -
        - -
          -
        • - - -

          Method Summary

          - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
          All Methods Instance Methods Concrete Methods 
          Modifier and TypeMethod and Description
          voidclose() 
          intgenerateOperationId() 
          intgenerateRequestId(bftsmart.tom.core.messages.TOMMessageType type) 
          bftsmart.communication.client.CommunicationSystemClientSidegetCommunicationSystem() 
          intgetProcessId() 
          intgetSession() 
          bftsmart.reconfiguration.ClientViewControllergetViewManager() 
          voidinit(int processId) -
          This method initializes the object - TODO: Ask if this method cannot be protected (compiles, but....)
          -
          voidinit(int processId, - java.lang.String configHome) 
          voidsendMessageToTargets(byte[] m, - int reqId, - int[] targets, - bftsmart.tom.core.messages.TOMMessageType type) 
          voidsendMessageToTargets(byte[] m, - int reqId, - int operationId, - int[] targets, - bftsmart.tom.core.messages.TOMMessageType type) 
          voidTOMulticast(byte[] m, - int reqId, - int operationId, - bftsmart.tom.core.messages.TOMMessageType reqType) 
          voidTOMulticast(byte[] m, - int reqId, - bftsmart.tom.core.messages.TOMMessageType reqType) -
          Multicast data to the group of replicas
          -
          voidTOMulticast(bftsmart.tom.core.messages.TOMMessage sm) -
          Multicast a TOMMessage to the group of replicas
          -
          -
            -
          • - - -

            Methods inherited from class java.lang.Object

            -clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
            • -
          -
            -
          • - - -

            Methods inherited from interface bftsmart.communication.client.ReplyReceiver

            -replyReceived
            • -
          -
          • -
        -
        • -
      -
      -
      -
        -
      • - -
          -
        • - - -

          Constructor Detail

          - - - -
            -
          • -

            TOMSender

            -
            public TOMSender()
            -
            Creates a new instance of TOMulticastSender - - TODO: This may really be empty?
            -
            • -
          -
          • -
        - -
          -
        • - - -

          Method Detail

          - - - -
            -
          • -

            close

            -
            public void close()
            -
            • -
          - - - -
            -
          • -

            getCommunicationSystem

            -
            public bftsmart.communication.client.CommunicationSystemClientSide getCommunicationSystem()
            -
            • -
          - - - -
            -
          • -

            getViewManager

            -
            public bftsmart.reconfiguration.ClientViewController getViewManager()
            -
            • -
          - - - -
            -
          • -

            init

            -
            public void init(int processId)
            -
            This method initializes the object - TODO: Ask if this method cannot be protected (compiles, but....)
            -
            -
            Parameters:
            -
            processId - ID of the process
            -
            -
            • -
          - - - -
            -
          • -

            init

            -
            public void init(int processId,
-                 java.lang.String configHome)
            -
            • -
          - - - -
            -
          • -

            getProcessId

            -
            public int getProcessId()
            -
            • -
          - - - -
            -
          • -

            generateRequestId

            -
            public int generateRequestId(bftsmart.tom.core.messages.TOMMessageType type)
            -
            • -
          - - - -
            -
          • -

            generateOperationId

            -
            public int generateOperationId()
            -
            • -
          - - - -
            -
          • -

            TOMulticast

            -
            public void TOMulticast(bftsmart.tom.core.messages.TOMMessage sm)
            -
            Multicast a TOMMessage to the group of replicas
            -
            -
            Parameters:
            -
            sm - Message to be multicast
            -
            -
            • -
          - - - -
            -
          • -

            TOMulticast

            -
            public void TOMulticast(byte[] m,
-                        int reqId,
-                        bftsmart.tom.core.messages.TOMMessageType reqType)
            -
            Multicast data to the group of replicas
            -
            -
            Parameters:
            -
            m - Data to be multicast
            -
            reqId - unique integer that identifies this request
            -
            reqType - TOM_NORMAL, TOM_READONLY or TOM_RECONFIGURATION
            -
            -
            • -
          - - - -
            -
          • -

            TOMulticast

            -
            public void TOMulticast(byte[] m,
-                        int reqId,
-                        int operationId,
-                        bftsmart.tom.core.messages.TOMMessageType reqType)
            -
            • -
          - - - -
            -
          • -

            sendMessageToTargets

            -
            public void sendMessageToTargets(byte[] m,
-                                 int reqId,
-                                 int[] targets,
-                                 bftsmart.tom.core.messages.TOMMessageType type)
            -
            • -
          - - - -
            -
          • -

            sendMessageToTargets

            -
            public void sendMessageToTargets(byte[] m,
-                                 int reqId,
-                                 int operationId,
-                                 int[] targets,
-                                 bftsmart.tom.core.messages.TOMMessageType type)
            -
            • -
          - - - -
            -
          • -

            getSession

            -
            public int getSession()
            -
            • -
          -
          • -
        -
        • -
      -
      -
      - - -
      - - -
      Skip navigation links
      - - - - -
      -
      - - - -
      - -
      -
      -
        -
      • Summary: 
        • -
      • Nested | 
        • -
      • Field | 
        • -
      • Constr | 
        • -
      • Method
        • -
      - -
      - - -
      - - - diff --git a/doc/bftsmart/tom/package-frame.html b/doc/bftsmart/tom/package-frame.html index 6b11d67aa..de8dd158a 100644 --- a/doc/bftsmart/tom/package-frame.html +++ b/doc/bftsmart/tom/package-frame.html @@ -2,9 +2,9 @@ - + bftsmart.tom - + @@ -19,7 +19,6 @@

      Classes

    • RequestContext
    • ServiceProxy
    • ServiceReplica
      • -
    • TOMSender
    diff --git a/doc/bftsmart/tom/package-summary.html b/doc/bftsmart/tom/package-summary.html index 82bdda53d..4a1323820 100644 --- a/doc/bftsmart/tom/package-summary.html +++ b/doc/bftsmart/tom/package-summary.html @@ -2,9 +2,9 @@ - + bftsmart.tom - + @@ -42,7 +42,7 @@
      @@ -120,12 +120,6 @@

      Package bftsmart.tom

      from the application and reply to the clients.
    - -TOMSender - -
    This class is used to multicast messages to replicas and receive replies.
    - - @@ -151,7 +145,7 @@

    Package bftsmart.tom

      diff --git a/doc/bftsmart/tom/package-tree.html b/doc/bftsmart/tom/package-tree.html index 44e243727..9733f63c8 100644 --- a/doc/bftsmart/tom/package-tree.html +++ b/doc/bftsmart/tom/package-tree.html @@ -2,9 +2,9 @@ - + bftsmart.tom Class Hierarchy - + @@ -42,7 +42,7 @@
      @@ -84,7 +84,7 @@

      Class Hierarchy

    • bftsmart.tom.ReplicaContext
    • bftsmart.tom.RequestContext
    • bftsmart.tom.ServiceReplica
      • -
    • bftsmart.tom.TOMSender (implements bftsmart.communication.client.ReplyReceiver) +
    • bftsmart.tom.core.TOMSender (implements java.lang.AutoCloseable, java.io.Closeable, bftsmart.communication.client.ReplyReceiver)
      • bftsmart.tom.ServiceProxy
          @@ -117,7 +117,7 @@

          Class Hierarchy

      diff --git a/doc/bftsmart/tom/server/BatchExecutable.html b/doc/bftsmart/tom/server/BatchExecutable.html index a93e3d91e..cb3568e87 100644 --- a/doc/bftsmart/tom/server/BatchExecutable.html +++ b/doc/bftsmart/tom/server/BatchExecutable.html @@ -2,9 +2,9 @@ - + BatchExecutable - + @@ -170,8 +170,8 @@

      executeBatch

      Execute a batch of requests.
      Parameters:
      -
      command -
      -
      msgCtx -
      +
      command - The batch of requests
      +
      msgCtx - The context associated to each request
      Returns:
      diff --git a/doc/bftsmart/tom/server/Executable.html b/doc/bftsmart/tom/server/Executable.html index e1dea335e..3d0e26439 100644 --- a/doc/bftsmart/tom/server/Executable.html +++ b/doc/bftsmart/tom/server/Executable.html @@ -2,9 +2,9 @@ - + Executable - + @@ -49,7 +49,7 @@
      • Frames
        • @@ -100,7 +100,7 @@

        Interface Executable

      • All Known Subinterfaces:
        -
        BatchExecutable, FIFOExecutable, SingleExecutable
        +
        BatchExecutable, SingleExecutable
        All Known Implementing Classes:
        @@ -201,7 +201,7 @@

        executeUnordered

        • Frames
          • diff --git a/doc/bftsmart/tom/server/FIFOExecutable.html b/doc/bftsmart/tom/server/FIFOExecutable.html deleted file mode 100644 index 2e621059e..000000000 --- a/doc/bftsmart/tom/server/FIFOExecutable.html +++ /dev/null @@ -1,265 +0,0 @@ - - - - - -FIFOExecutable - - - - - - - - -
          - - -
          Skip navigation links
          - - - - -
          -
          - - - -
          - -
          -
          -
            -
          • Summary: 
            • -
          • Nested | 
            • -
          • Field | 
            • -
          • Constr | 
            • -
          • Method
            • -
          -
            -
          • Detail: 
            • -
          • Field | 
            • -
          • Constr | 
            • -
          • Method
            • -
          -
          - - -
          - - -
          -
          bftsmart.tom.server
          -

          Interface FIFOExecutable

          -
          -
          -
          -
            -
          • -
            -
            All Superinterfaces:
            -
            Executable, SingleExecutable
            -
            -
            -
            -
            public interface FIFOExecutable
-extends SingleExecutable
            -
            Executables that implement this interface will get requests - delivered in FIFO order.
            -
            • -
          -
          -
          - -
          -
          -
            -
          • - -
              -
            • - - -

              Method Detail

              - - - -
                -
              • -

                executeOrderedFIFO

                -
                byte[] executeOrderedFIFO(byte[] command,
-                          MessageContext msgCtx,
-                          int clientId,
-                          int operationId)
                -
                • -
              - - - -
                -
              • -

                executeUnorderedFIFO

                -
                byte[] executeUnorderedFIFO(byte[] command,
-                            MessageContext msgCtx,
-                            int clientId,
-                            int operationId)
                -
                • -
              -
              • -
            -
            • -
          -
          -
          - - -
          - - -
          Skip navigation links
          - - - - -
          -
          - - - -
          - -
          -
          -
            -
          • Summary: 
            • -
          • Nested | 
            • -
          • Field | 
            • -
          • Constr | 
            • -
          • Method
            • -
          -
            -
          • Detail: 
            • -
          • Field | 
            • -
          • Constr | 
            • -
          • Method
            • -
          -
          - - -
          - - - diff --git a/doc/bftsmart/tom/server/Recoverable.html b/doc/bftsmart/tom/server/Recoverable.html index 7deefcf1e..3efbd916c 100644 --- a/doc/bftsmart/tom/server/Recoverable.html +++ b/doc/bftsmart/tom/server/Recoverable.html @@ -2,9 +2,9 @@ - + Recoverable - + @@ -48,7 +48,7 @@
          @@ -142,8 +142,9 @@

          Method Summary

          void -noOp(int CID, - MessageContext msgCtx) +noOp(int CID, + byte[][] operations, + MessageContext[] msgCtx)
          This method is invoked by ServiceReplica to indicate that a consensus instance finished without delivering anything to the application (e.g., an instance only decided a single reconfiguration operation.
          @@ -160,7 +161,9 @@

          Method Summary

          void -setReplicaContext(ReplicaContext replicaContext)  +setReplicaContext(ReplicaContext replicaContext) +
          Sets the replica context
          + int @@ -190,6 +193,11 @@

          Method Detail

        • setReplicaContext

          void setReplicaContext(ReplicaContext replicaContext)
          +
          Sets the replica context
          +
          +
          Parameters:
          +
          replicaContext - The replica context
          +
        @@ -267,14 +275,15 @@

        Op

      - +

      • noOp

        void noOp(int CID,
-          MessageContext msgCtx)
        + byte[][] operations, + MessageContext[] msgCtx)
        This method is invoked by ServiceReplica to indicate that a consensus instance finished without delivering anything to the application (e.g., an instance only decided a single reconfiguration operation. or an instance where the client @@ -285,6 +294,7 @@

        noOp

        Parameters:
        CID - the consensus instance where the aforementioned condition occurred
        +
        operations - Operations decided in CID
        msgCtx - Message context associated with the consensus instance. furthermore msgCtx.getConsensusId() will be equal to CID.
        @@ -317,7 +327,7 @@

        noOp

          diff --git a/doc/bftsmart/tom/server/Replier.html b/doc/bftsmart/tom/server/Replier.html index 2c26e36db..e0a000308 100644 --- a/doc/bftsmart/tom/server/Replier.html +++ b/doc/bftsmart/tom/server/Replier.html @@ -2,9 +2,9 @@ - + Replier - + @@ -101,8 +101,8 @@

          Interface Replier


          public interface Replier
          -
          Objects that implement this interface can be passed to the ServiceReplica - and manage replies in a custom manner.
          +
          Provides support for building custom reply management + to be used in the ServiceReplica.
        @@ -124,11 +124,15 @@

        Method Summary

        void manageReply(bftsmart.tom.core.messages.TOMMessage request, - MessageContext msgCtx)  + MessageContext msgCtx)         +
        Given an executed request, send it to a target
        + void -setReplicaContext(ReplicaContext rc)  +setReplicaContext(ReplicaContext replicaContext) +
        Sets the replica context
        +
        • @@ -151,7 +155,12 @@

        Method Detail

        • setReplicaContext

          -
          void setReplicaContext(ReplicaContext rc)
          +
          void setReplicaContext(ReplicaContext replicaContext)
          +
          Sets the replica context
          +
          +
          Parameters:
          +
          replicaContext - The replica context
          +
        @@ -162,6 +171,12 @@

        setReplicaContext

        manageReply

        void manageReply(bftsmart.tom.core.messages.TOMMessage request,
                  MessageContext msgCtx)
        +
        Given an executed request, send it to a target
        +
        +
        Parameters:
        +
        request - The executed request
        +
        msgCtx - The message context associated to the request
        +
      diff --git a/doc/bftsmart/tom/server/RequestVerifier.html b/doc/bftsmart/tom/server/RequestVerifier.html index 25934411c..6c762532b 100644 --- a/doc/bftsmart/tom/server/RequestVerifier.html +++ b/doc/bftsmart/tom/server/RequestVerifier.html @@ -2,9 +2,9 @@ - + RequestVerifier - + @@ -127,7 +127,9 @@

      Method Summary

      boolean -isValidRequest(byte[] request)  +isValidRequest(bftsmart.tom.core.messages.TOMMessage request) +
      Given a request, validated it
      + @@ -144,13 +146,20 @@

      Method Summary

      Method Detail

      - +

      • isValidRequest

        -
        boolean isValidRequest(byte[] request)
        +
        boolean isValidRequest(bftsmart.tom.core.messages.TOMMessage request)
        +
        Given a request, validated it
        +
        +
        Parameters:
        +
        request - The request to be validated
        +
        Returns:
        +
        true if the request is valid, false otherwise
        +
      diff --git a/doc/bftsmart/tom/server/SingleExecutable.html b/doc/bftsmart/tom/server/SingleExecutable.html index e66af1ec2..457894aff 100644 --- a/doc/bftsmart/tom/server/SingleExecutable.html +++ b/doc/bftsmart/tom/server/SingleExecutable.html @@ -2,9 +2,9 @@ - + SingleExecutable - + @@ -103,10 +103,6 @@

      Interface SingleExecutable<
      Executable
      -
      All Known Subinterfaces:
      -
      FIFOExecutable
      -
      -
      All Known Implementing Classes:
      DefaultSingleRecoverable
      diff --git a/doc/bftsmart/tom/server/defaultservices/CommandsInfo.html b/doc/bftsmart/tom/server/defaultservices/CommandsInfo.html deleted file mode 100644 index 342150195..000000000 --- a/doc/bftsmart/tom/server/defaultservices/CommandsInfo.html +++ /dev/null @@ -1,404 +0,0 @@ - - - - - -CommandsInfo - - - - - - - - -
      - - -
      Skip navigation links
      - - - - -
      -
      - - - -
      - -
      -
      - - -
      - - -
      - - -
      -
      bftsmart.tom.server.defaultservices
      -

      Class CommandsInfo

      -
      -
      -
        -
      • java.lang.Object
        • -
      • -
          -
        • bftsmart.tom.server.defaultservices.CommandsInfo
          • -
        -
        • -
      -
      -
        -
      • -
        -
        All Implemented Interfaces:
        -
        java.io.Externalizable, java.io.Serializable
        -
        -
        -
        -
        public class CommandsInfo
-extends java.lang.Object
-implements java.io.Externalizable
        -
        -
        See Also:
        -
        Serialized Form
        -
        -
        • -
      -
      -
      -
        -
      • - - - - - -
          -
        • - - -

          Method Summary

          - - - - - - - - - - - - - - - - - - - - - - -
          All Methods Instance Methods Concrete Methods 
          Modifier and TypeMethod and Description
          booleanequals(java.lang.Object obj) 
          inthashCode() 
          voidreadExternal(java.io.ObjectInput in) 
          voidwriteExternal(java.io.ObjectOutput out) 
          -
            -
          • - - -

            Methods inherited from class java.lang.Object

            -clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait
            • -
          -
          • -
        -
        • -
      -
      -
      -
        -
      • - -
          -
        • - - -

          Field Detail

          - - - -
            -
          • -

            commands

            -
            public byte[][] commands
            -
            • -
          - - - - -
          • -
        - -
          -
        • - - -

          Constructor Detail

          - - - -
            -
          • -

            CommandsInfo

            -
            public CommandsInfo()
            -
            • -
          - - - -
            -
          • -

            CommandsInfo

            -
            public CommandsInfo(byte[][] commands,
-                    MessageContext[] msgCtx)
            -
            • -
          -
          • -
        - -
          -
        • - - -

          Method Detail

          - - - -
            -
          • -

            equals

            -
            public boolean equals(java.lang.Object obj)
            -
            -
            Overrides:
            -
            equals in class java.lang.Object
            -
            -
            • -
          - - - -
            -
          • -

            hashCode

            -
            public int hashCode()
            -
            -
            Overrides:
            -
            hashCode in class java.lang.Object
            -
            -
            • -
          - - - -
            -
          • -

            writeExternal

            -
            public void writeExternal(java.io.ObjectOutput out)
-                   throws java.io.IOException
            -
            -
            Specified by:
            -
            writeExternal in interface java.io.Externalizable
            -
            Throws:
            -
            java.io.IOException
            -
            -
            • -
          - - - -
            -
          • -

            readExternal

            -
            public void readExternal(java.io.ObjectInput in)
-                  throws java.io.IOException,
-                         java.lang.ClassNotFoundException
            -
            -
            Specified by:
            -
            readExternal in interface java.io.Externalizable
            -
            Throws:
            -
            java.io.IOException
            -
            java.lang.ClassNotFoundException
            -
            -
            • -
          -
          • -
        -
        • -
      -
      -
      - - -
      - - -
      Skip navigation links
      - - - - -
      -
      - - - -
      - -
      -
      - - -
      - - -
      - - - diff --git a/doc/bftsmart/tom/server/defaultservices/DefaultApplicationState.html b/doc/bftsmart/tom/server/defaultservices/DefaultApplicationState.html deleted file mode 100644 index 10cf48aaf..000000000 --- a/doc/bftsmart/tom/server/defaultservices/DefaultApplicationState.html +++ /dev/null @@ -1,704 +0,0 @@ - - - - - -DefaultApplicationState - - - - - - - - -
      - - -
      Skip navigation links
      - - - - -
      -
      - - - -
      - -
      -
      - - -
      - - -
      - - -
      -
      bftsmart.tom.server.defaultservices
      -

      Class DefaultApplicationState

      -
      -
      -
        -
      • java.lang.Object
        • -
      • -
          -
        • bftsmart.tom.server.defaultservices.DefaultApplicationState
          • -
        -
        • -
      -
      -
        -
      • -
        -
        All Implemented Interfaces:
        -
        bftsmart.statemanagement.ApplicationState, java.io.Serializable
        -
        -
        -
        -
        public class DefaultApplicationState
-extends java.lang.Object
-implements bftsmart.statemanagement.ApplicationState
        -
        This class represents a state transfered from a replica to another. The state associated with the last - checkpoint together with all the batches of messages received do far, comprises the sender's - current state
        -
        -
        See Also:
        -
        Serialized Form
        -
        -
        • -
      -
      -
      -
        -
      • - -
          -
        • - - -

          Field Summary

          - - - - - - - - - - - - - - - - - - - - - - -
          Fields 
          Modifier and TypeField and Description
          protected booleanhasState 
          protected intlastCID 
          protected byte[]state 
          protected byte[]stateHash 
          -
          • -
        - -
          -
        • - - -

          Constructor Summary

          - - - - - - - - - - - - - - -
          Constructors 
          Constructor and Description
          DefaultApplicationState() -
          Constructs a TansferableState - This constructor should be used when there isn't a valid state to construct the object with
          -
          DefaultApplicationState(CommandsInfo[] messageBatches, - byte[] logHash, - int lastCheckpointCID, - int lastCID, - byte[] state, - byte[] stateHash, - int pid) 
          DefaultApplicationState(CommandsInfo[] messageBatches, - int lastCheckpointCID, - int lastCID, - byte[] state, - byte[] stateHash, - int pid) -
          Constructs a TansferableState - This constructor should be used when there is a valid state to construct the object with
          -
          -
          • -
        - -
          -
        • - - -

          Method Summary

          - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
          All Methods Instance Methods Concrete Methods 
          Modifier and TypeMethod and Description
          booleanequals(java.lang.Object obj) -
          This method MUST be implemented.
          -
          bftsmart.tom.leaderchange.CertifiedDecisiongetCertifiedDecision(bftsmart.reconfiguration.ServerViewController controller) -
          Retrieves the certified decision for the last consensus present in this object
          -
          intgetLastCheckpointCID() -
          Retrieves the consensus ID for the last checkpoint
          -
          intgetLastCID() -
          Retrieves the consensus ID for the last messages batch delivered to the application
          -
          CommandsInfogetMessageBatch(int cid) -
          Retrieves the specified batch of messages
          -
          CommandsInfo[]getMessageBatches() -
          Retrieves all batches of messages
          -
          byte[]getSerializedState() -
          Byte array that must be a representation of the application state
          -
          byte[]getState() -
          Retrieves the state associated with the last checkpoint
          -
          byte[]getStateHash() -
          Retrieves the hash of the state associated with the last checkpoint
          -
          inthashCode() -
          This method MUST be implemented.
          -
          booleanhasState() -
          Indicates if the TransferableState object has a valid state
          -
          voidsetMessageBatches(CommandsInfo[] messageBatches) 
          voidsetSerializedState(byte[] state) -
          Sets a byte array that must be a representation of the application state
          -
          voidsetState(byte[] state) -
          Sets the state associated with the last checkpoint
          -
          -
            -
          • - - -

            Methods inherited from class java.lang.Object

            -clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait
            • -
          -
          • -
        -
        • -
      -
      -
      -
        -
      • - -
          -
        • - - -

          Field Detail

          - - - -
            -
          • -

            state

            -
            protected byte[] state
            -
            • -
          - - - -
            -
          • -

            stateHash

            -
            protected byte[] stateHash
            -
            • -
          - - - -
            -
          • -

            lastCID

            -
            protected int lastCID
            -
            • -
          - - - -
            -
          • -

            hasState

            -
            protected boolean hasState
            -
            • -
          -
          • -
        - -
          -
        • - - -

          Constructor Detail

          - - - -
            -
          • -

            DefaultApplicationState

            -
            public DefaultApplicationState(CommandsInfo[] messageBatches,
-                               int lastCheckpointCID,
-                               int lastCID,
-                               byte[] state,
-                               byte[] stateHash,
-                               int pid)
            -
            Constructs a TansferableState - This constructor should be used when there is a valid state to construct the object with
            -
            -
            Parameters:
            -
            messageBatches - Batches received since the last checkpoint.
            -
            state - State associated with the last checkpoint
            -
            stateHash - Hash of the state associated with the last checkpoint
            -
            -
            • -
          - - - -
            -
          • -

            DefaultApplicationState

            -
            public DefaultApplicationState(CommandsInfo[] messageBatches,
-                               byte[] logHash,
-                               int lastCheckpointCID,
-                               int lastCID,
-                               byte[] state,
-                               byte[] stateHash,
-                               int pid)
            -
            • -
          - - - -
            -
          • -

            DefaultApplicationState

            -
            public DefaultApplicationState()
            -
            Constructs a TansferableState - This constructor should be used when there isn't a valid state to construct the object with
            -
            • -
          -
          • -
        - -
          -
        • - - -

          Method Detail

          - - - -
            -
          • -

            setSerializedState

            -
            public void setSerializedState(byte[] state)
            -
            Description copied from interface: bftsmart.statemanagement.ApplicationState
            -
            Sets a byte array that must be a representation of the application state
            -
            -
            Specified by:
            -
            setSerializedState in interface bftsmart.statemanagement.ApplicationState
            -
            Parameters:
            -
            state - a byte array that must be a representation of the application state
            -
            -
            • -
          - - - -
            -
          • -

            getSerializedState

            -
            public byte[] getSerializedState()
            -
            Description copied from interface: bftsmart.statemanagement.ApplicationState
            -
            Byte array that must be a representation of the application state
            -
            -
            Specified by:
            -
            getSerializedState in interface bftsmart.statemanagement.ApplicationState
            -
            Returns:
            -
            A byte array that must be a representation of the application state
            -
            -
            • -
          - - - -
            -
          • -

            hasState

            -
            public boolean hasState()
            -
            Indicates if the TransferableState object has a valid state
            -
            -
            Specified by:
            -
            hasState in interface bftsmart.statemanagement.ApplicationState
            -
            Returns:
            -
            true if it has a valid state, false otherwise
            -
            -
            • -
          - - - -
            -
          • -

            getLastCID

            -
            public int getLastCID()
            -
            Retrieves the consensus ID for the last messages batch delivered to the application
            -
            -
            Specified by:
            -
            getLastCID in interface bftsmart.statemanagement.ApplicationState
            -
            Returns:
            -
            Consensus ID for the last messages batch delivered to the application
            -
            -
            • -
          - - - -
            -
          • -

            getCertifiedDecision

            -
            public bftsmart.tom.leaderchange.CertifiedDecision getCertifiedDecision(bftsmart.reconfiguration.ServerViewController controller)
            -
            Retrieves the certified decision for the last consensus present in this object
            -
            -
            Specified by:
            -
            getCertifiedDecision in interface bftsmart.statemanagement.ApplicationState
            -
            Parameters:
            -
            controller -
            -
            Returns:
            -
            The certified decision for the last consensus present in this object
            -
            -
            • -
          - - - -
            -
          • -

            getState

            -
            public byte[] getState()
            -
            Retrieves the state associated with the last checkpoint
            -
            -
            Returns:
            -
            State associated with the last checkpoint
            -
            -
            • -
          - - - -
            -
          • -

            getStateHash

            -
            public byte[] getStateHash()
            -
            Retrieves the hash of the state associated with the last checkpoint
            -
            -
            Specified by:
            -
            getStateHash in interface bftsmart.statemanagement.ApplicationState
            -
            Returns:
            -
            Hash of the state associated with the last checkpoint
            -
            -
            • -
          - - - -
            -
          • -

            setState

            -
            public void setState(byte[] state)
            -
            Sets the state associated with the last checkpoint
            -
            -
            Parameters:
            -
            state - State associated with the last checkpoint
            -
            -
            • -
          - - - -
            -
          • -

            getMessageBatches

            -
            public CommandsInfo[] getMessageBatches()
            -
            Retrieves all batches of messages
            -
            -
            Returns:
            -
            Batch of messages
            -
            -
            • -
          - - - -
            -
          • -

            setMessageBatches

            -
            public void setMessageBatches(CommandsInfo[] messageBatches)
            -
            • -
          - - - -
            -
          • -

            getMessageBatch

            -
            public CommandsInfo getMessageBatch(int cid)
            -
            Retrieves the specified batch of messages
            -
            -
            Parameters:
            -
            cid - Consensus ID associated with the batch to be fetched
            -
            Returns:
            -
            The batch of messages associated with the batch correspondent consensus ID
            -
            -
            • -
          - - - -
            -
          • -

            getLastCheckpointCID

            -
            public int getLastCheckpointCID()
            -
            Retrieves the consensus ID for the last checkpoint
            -
            -
            Returns:
            -
            Consensus ID for the last checkpoint, or -1 if no checkpoint was yet executed
            -
            -
            • -
          - - - -
            -
          • -

            equals

            -
            public boolean equals(java.lang.Object obj)
            -
            Description copied from interface: bftsmart.statemanagement.ApplicationState
            -
            This method MUST be implemented. However, the attribute returned by getSerializedState() - should be ignored, and getStateHash() should be used instead
            -
            -
            Specified by:
            -
            equals in interface bftsmart.statemanagement.ApplicationState
            -
            Overrides:
            -
            equals in class java.lang.Object
            -
            -
            • -
          - - - -
            -
          • -

            hashCode

            -
            public int hashCode()
            -
            Description copied from interface: bftsmart.statemanagement.ApplicationState
            -
            This method MUST be implemented. However, the attribute returned by getSerializedState() - should be ignored, and getStateHash() should be used instead
            -
            -
            Specified by:
            -
            hashCode in interface bftsmart.statemanagement.ApplicationState
            -
            Overrides:
            -
            hashCode in class java.lang.Object
            -
            -
            • -
          -
          • -
        -
        • -
      -
      -
      - - -
      - - -
      Skip navigation links
      - - - - -
      -
      - - - -
      - -
      -
      - - -
      - - -
      - - - diff --git a/doc/bftsmart/tom/server/defaultservices/DefaultRecoverable.html b/doc/bftsmart/tom/server/defaultservices/DefaultRecoverable.html index 119caef1d..bad4ffaea 100644 --- a/doc/bftsmart/tom/server/defaultservices/DefaultRecoverable.html +++ b/doc/bftsmart/tom/server/defaultservices/DefaultRecoverable.html @@ -2,9 +2,9 @@ - + DefaultRecoverable - + @@ -18,7 +18,7 @@ catch(err) { } //--> -var methods = {"i0":6,"i1":6,"i2":10,"i3":10,"i4":10,"i5":6,"i6":10,"i7":10,"i8":6,"i9":10,"i10":10,"i11":10,"i12":10,"i13":10}; +var methods = {"i0":6,"i1":6,"i2":10,"i3":10,"i4":6,"i5":10,"i6":10,"i7":6,"i8":10,"i9":10,"i10":10,"i11":10}; var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],4:["t3","Abstract Methods"],8:["t4","Concrete Methods"]}; var altColor = "altColor"; var rowColor = "rowColor"; @@ -135,7 +135,9 @@

      Constructor Summary

      Constructor and Description -DefaultRecoverable()  +DefaultRecoverable() +
      Constructor
      + @@ -154,44 +156,47 @@

      Method Summary

      abstract byte[][] -appExecuteBatch(byte[][] commands, - MessageContext[] msgCtxs)  +appExecuteBatch(byte[][] commands, + MessageContext[] msgCtxs, + boolean fromConsensus) +
      Execute a batch of ordered requests
      + abstract byte[] appExecuteUnordered(byte[] command, - MessageContext msgCtx)  + MessageContext msgCtx) +
      Execute an unordered request
      + -byte[] -computeHash(byte[] data)  - - byte[][] executeBatch(byte[][] commands, MessageContext[] msgCtxs)
      Execute a batch of requests.
      - + byte[] executeUnordered(byte[] command, MessageContext msgCtx)
      Method called to execute a request totally ordered.
      - + abstract byte[] -getSnapshot()  +getSnapshot() +
      Returns a serialized snapshot of the application state
      + - + bftsmart.statemanagement.ApplicationState getState(int cid, boolean sendState)
      This method should return a representation of the application state
      - + bftsmart.statemanagement.StateManager getStateManager()
      Recoverers implementing this interface will have to chose among @@ -199,20 +204,23 @@

      Method Summary

      StandardStateManager.
      - + abstract void -installSnapshot(byte[] state)  +installSnapshot(byte[] state) +
      Given a snapshot received from the state transfer protocol, install it
      + - + void -noOp(int CID, - MessageContext msgCtx) +noOp(int CID, + byte[][] operations, + MessageContext[] msgCtxs)
      This method is invoked by ServiceReplica to indicate that a consensus instance finished without delivering anything to the application (e.g., an instance only decided a single reconfiguration operation.
      - + void Op(int CID, byte[] requests, @@ -221,18 +229,13 @@

      Method Summary

      decided in a particular consensus instance.
      - + void -saveCommands(byte[][] commands, - MessageContext[] msgCtx) -
      Write commands to log file
      +setReplicaContext(ReplicaContext replicaContext) +
      Sets the replica context
      - -void -setReplicaContext(ReplicaContext replicaContext)  - - + int setState(bftsmart.statemanagement.ApplicationState recvState)
      Sets the state to the representation obtained in the state transfer protocol
      @@ -267,6 +270,7 @@

      Constructor Detail

    • DefaultRecoverable

      public DefaultRecoverable()
      +
      Constructor
    @@ -290,32 +294,10 @@

    executeBatch

    Specified by:
    executeBatch in interface BatchExecutable
    -
    Returns:
    -
    - - - - - -
      -
    • -

      computeHash

      -
      public final byte[] computeHash(byte[] data)
      -
      • -
    - - - -
      -
    • -

      saveCommands

      -
      public void saveCommands(byte[][] commands,
-                         MessageContext[] msgCtx)
      -
      Write commands to log file
      -
      Parameters:
      -
      commands - array of commands. Each command is an array of bytes
      -
      msgCtx -
      +
      commands - The batch of requests
      +
      msgCtxs - The context associated to each request
      +
      Returns:
    @@ -366,9 +348,13 @@

    setState

  • setReplicaContext

    public void setReplicaContext(ReplicaContext replicaContext)
    +
    Description copied from interface: Recoverable
    +
    Sets the replica context
    Specified by:
    setReplicaContext in interface Recoverable
    +
    Parameters:
    +
    replicaContext - The replica context
    • @@ -443,15 +429,16 @@

    Op

    - +

    • noOp

      public void noOp(int CID,
-                 MessageContext msgCtx)
      -
      Description copied from interface: Recoverable
      + byte[][] operations, + MessageContext[] msgCtxs) +
      Description copied from interface: Recoverable
      This method is invoked by ServiceReplica to indicate that a consensus instance finished without delivering anything to the application (e.g., an instance only decided a single reconfiguration operation. or an instance where the client @@ -461,10 +448,11 @@

      noOp

      logs used within the state transfer, but never deliver it to the application
      Specified by:
      -
      noOp in interface Recoverable
      +
      noOp in interface Recoverable
      Parameters:
      CID - the consensus instance where the aforementioned condition occurred
      -
      msgCtx - Message context associated with the consensus instance. furthermore +
      operations - Operations decided in CID
      +
      msgCtxs - Message context associated with the consensus instance. furthermore msgCtx.getConsensusId() will be equal to CID.
      • @@ -476,6 +464,11 @@

      noOp

    • installSnapshot

      public abstract void installSnapshot(byte[] state)
      +
      Given a snapshot received from the state transfer protocol, install it
      +
      +
      Parameters:
      +
      state - The serialized snapshot
      +
    @@ -485,16 +478,31 @@

    installSnapshot

  • getSnapshot

    public abstract byte[] getSnapshot()
    +
    Returns a serialized snapshot of the application state
    +
    +
    Returns:
    +
    A serialized snapshot of the application state
    +
    • -     +

    • appExecuteBatch

      public abstract byte[][] appExecuteBatch(byte[][] commands,
-                                         MessageContext[] msgCtxs)
      + MessageContext[] msgCtxs, + boolean fromConsensus) +
      Execute a batch of ordered requests
      +
      +
      Parameters:
      +
      commands - The batch of requests
      +
      msgCtxs - The context associated to each request
      +
      fromConsensus - true if the request arrived from a consensus execution, false if it arrives from the state transfer protocol
      +
      Returns:
      +
      the respective replies for each request
      +
    @@ -505,6 +513,14 @@

    appExecuteBatch

    appExecuteUnordered

    public abstract byte[] appExecuteUnordered(byte[] command,
                                            MessageContext msgCtx)
    +
    Execute an unordered request
    +
    +
    Parameters:
    +
    command - The unordered request
    +
    msgCtx - The context associated to the request
    +
    Returns:
    +
    the reply for the request issued by the client
    +
    diff --git a/doc/bftsmart/tom/server/defaultservices/DefaultSingleRecoverable.html b/doc/bftsmart/tom/server/defaultservices/DefaultSingleRecoverable.html index 2d49684b2..6a57a4c6b 100644 --- a/doc/bftsmart/tom/server/defaultservices/DefaultSingleRecoverable.html +++ b/doc/bftsmart/tom/server/defaultservices/DefaultSingleRecoverable.html @@ -2,9 +2,9 @@ - + DefaultSingleRecoverable - + @@ -18,7 +18,7 @@ catch(err) { } //--> -var methods = {"i0":6,"i1":6,"i2":10,"i3":10,"i4":10,"i5":6,"i6":10,"i7":10,"i8":10,"i9":6,"i10":10,"i11":10,"i12":10,"i13":10}; +var methods = {"i0":6,"i1":6,"i2":10,"i3":10,"i4":6,"i5":10,"i6":10,"i7":6,"i8":10,"i9":10,"i10":10,"i11":10}; var tabs = {65535:["t0","All Methods"],2:["t2","Instance Methods"],4:["t3","Abstract Methods"],8:["t4","Concrete Methods"]}; var altColor = "altColor"; var rowColor = "rowColor"; @@ -174,43 +174,45 @@

    Method Summary

    abstract byte[] appExecuteOrdered(byte[] command, - MessageContext msgCtx)  + MessageContext msgCtx)     +
    Execute a batch of ordered requests
    + abstract byte[] appExecuteUnordered(byte[] command, - MessageContext msgCtx)  + MessageContext msgCtx) +
    Execute an unordered request
    + byte[] -computeHash(byte[] data)  - - -byte[] executeOrdered(byte[] command, MessageContext msgCtx)
    Method called to execute a request totally ordered.
    - + byte[] executeUnordered(byte[] command, MessageContext msgCtx)
    Method called to execute a request totally ordered.
    - + abstract byte[] -getSnapshot()  +getSnapshot() +
    Returns a serialized snapshot of the application state
    + - + bftsmart.statemanagement.ApplicationState getState(int cid, boolean sendState)
    This method should return a representation of the application state
    - + bftsmart.statemanagement.StateManager getStateManager()
    Recoverers implementing this interface will have to chose among @@ -218,24 +220,23 @@

    Method Summary

    StandardStateManager.
    - -protected void -initLog()  - - + abstract void -installSnapshot(byte[] state)  +installSnapshot(byte[] state) +
    Given a snapshot received from the state transfer protocol, install it
    + - + void -noOp(int CID, - MessageContext msgCtx) +noOp(int CID, + byte[][] operations, + MessageContext[] msgCtx)
    This method is invoked by ServiceReplica to indicate that a consensus instance finished without delivering anything to the application (e.g., an instance only decided a single reconfiguration operation.
    - + void Op(int CID, byte[] requests, @@ -244,11 +245,13 @@

    Method Summary

    decided in a particular consensus instance.
    - + void -setReplicaContext(ReplicaContext replicaContext)  +setReplicaContext(ReplicaContext replicaContext) +
    Sets the replica context
    + - + int setState(bftsmart.statemanagement.ApplicationState recvState)
    Sets the state to the representation obtained in the state transfer protocol
    @@ -339,15 +342,6 @@

    executeOrdered

    - - - -
      -
    • -

      computeHash

      -
      public final byte[] computeHash(byte[] data)
      -
      • -
    @@ -395,9 +389,13 @@

    setState

  • setReplicaContext

    public void setReplicaContext(ReplicaContext replicaContext)
    +
    Description copied from interface: Recoverable
    +
    Sets the replica context
    Specified by:
    setReplicaContext in interface Recoverable
    +
    Parameters:
    +
    replicaContext - The replica context
    • @@ -421,15 +419,6 @@

    getStateManager

    - - - -
      -
    • -

      initLog

      -
      protected void initLog()
      -
      • -
    @@ -481,15 +470,16 @@

    Op

    - +

    • noOp

      public void noOp(int CID,
-                 MessageContext msgCtx)
      -
      Description copied from interface: Recoverable
      + byte[][] operations, + MessageContext[] msgCtx) +
      Description copied from interface: Recoverable
      This method is invoked by ServiceReplica to indicate that a consensus instance finished without delivering anything to the application (e.g., an instance only decided a single reconfiguration operation. or an instance where the client @@ -499,9 +489,10 @@

      noOp

      logs used within the state transfer, but never deliver it to the application
      Specified by:
      -
      noOp in interface Recoverable
      +
      noOp in interface Recoverable
      Parameters:
      CID - the consensus instance where the aforementioned condition occurred
      +
      operations - Operations decided in CID
      msgCtx - Message context associated with the consensus instance. furthermore msgCtx.getConsensusId() will be equal to CID.
      @@ -514,6 +505,11 @@

      noOp

    • installSnapshot

      public abstract void installSnapshot(byte[] state)
      +
      Given a snapshot received from the state transfer protocol, install it
      +
      +
      Parameters:
      +
      state - The serialized snapshot
      +
    @@ -523,6 +519,11 @@

    installSnapshot

  • getSnapshot

    public abstract byte[] getSnapshot()
    +
    Returns a serialized snapshot of the application state
    +
    +
    Returns:
    +
    A serialized snapshot of the application state
    +
    •  @@ -533,6 +534,14 @@

    getSnapshot

    appExecuteOrdered

    public abstract byte[] appExecuteOrdered(byte[] command,
                                          MessageContext msgCtx)
    +
    Execute a batch of ordered requests
    +
    +
    Parameters:
    +
    command - The ordered request
    +
    msgCtx - The context associated to each request
    +
    Returns:
    +
    the reply for the request issued by the client
    +
    @@ -543,6 +552,14 @@

    appExecuteOrdered

    appExecuteUnordered

    public abstract byte[] appExecuteUnordered(byte[] command,
                                            MessageContext msgCtx)
    +
    Execute an unordered request
    +
    +
    Parameters:
    +
    command - The unordered request
    +
    msgCtx - The context associated to the request
    +
    Returns:
    +
    the reply for the request issued by the client
    +
    diff --git a/doc/bftsmart/tom/server/defaultservices/DiskStateLog.html b/doc/bftsmart/tom/server/defaultservices/DiskStateLog.html deleted file mode 100644 index b7ecfcd29..000000000 --- a/doc/bftsmart/tom/server/defaultservices/DiskStateLog.html +++ /dev/null @@ -1,469 +0,0 @@ - - - - - -DiskStateLog - - - - - - - - -
    - - -
    Skip navigation links
    - - - - -
    -
    - - - -
    - -
    -
    - - -
    - - -
    - - -
    -
    bftsmart.tom.server.defaultservices
    -

    Class DiskStateLog

    -
    -
    - -
    -
      -
    • -
      -
      -
      public class DiskStateLog
-extends StateLog
      -
      • -
    -
    -
    - -
    -
    -
      -
    • - -
        -
      • - - -

        Field Detail

        - - - -
          -
        • -

          DEFAULT_DIR

          -
          public static final java.lang.String DEFAULT_DIR
          -
          • -
        -
        • -
      - -
        -
      • - - -

        Constructor Detail

        - - - -
          -
        • -

          DiskStateLog

          -
          public DiskStateLog(int id,
-                    byte[] initialState,
-                    byte[] initialHash,
-                    boolean isToLog,
-                    boolean syncLog,
-                    boolean syncCkp)
          -
          • -
        -
        • -
      - -
        -
      • - - -

        Method Detail

        - - - -
          -
        • -

          addMessageBatch

          -
          public void addMessageBatch(byte[][] commands,
-                            MessageContext[] msgCtx,
-                            int consensusId)
          -
          Adds a message batch to the log. This batches should be added to the log - in the same order in which they are delivered to the application. Only - the 'k' batches received after the last checkpoint are supposed to be - kept
          -
          -
          Overrides:
          -
          addMessageBatch in class StateLog
          -
          Parameters:
          -
          commands - The batch of messages to be kept.
          -
          consensusId -
          -
          msgCtx - The message contexts related to the commands
          -
          -
          • -
        - - - -
          -
        • -

          newCheckpoint

          -
          public void newCheckpoint(byte[] state,
-                          byte[] stateHash,
-                          int consensusId)
          -
          Description copied from class: StateLog
          -
          Sets the state associated with the last checkpoint, and updates the consensus ID associated with it
          -
          -
          Overrides:
          -
          newCheckpoint in class StateLog
          -
          Parameters:
          -
          state - State associated with the last checkpoint
          -
          -
          • -
        - - - -
          -
        • -

          getApplicationState

          -
          public DefaultApplicationState getApplicationState(int cid,
-                                                   boolean sendState)
          -
          Constructs a TransferableState using this log information
          -
          -
          Overrides:
          -
          getApplicationState in class StateLog
          -
          Parameters:
          -
          cid - Consensus ID correspondent to desired state
          -
          sendState -
          -
          Returns:
          -
          TransferableState Object containing this log information
          -
          -
          • -
        - - - -
          -
        • -

          transferApplicationState

          -
          public void transferApplicationState(java.nio.channels.SocketChannel sChannel,
-                                     int cid)
          -
          • -
        - - - -
          -
        • -

          setLastCID

          -
          public void setLastCID(int cid,
-                       int checkpointPeriod,
-                       int checkpointPortion)
          -
          • -
        - - - -
          -
        • -

          update

          -
          public void update(DefaultApplicationState transState)
          -
          Updates this log, according to the information contained in the - TransferableState object
          -
          -
          Overrides:
          -
          update in class StateLog
          -
          Parameters:
          -
          transState - TransferableState object containing the information which is - used to updated this log
          -
          -
          • -
        - - - -
          -
        • -

          loadDurableState

          -
          protected bftsmart.statemanagement.ApplicationState loadDurableState()
          -
          • -
        -
        • -
      -
      • -
    -
    -
    - - -
    - - -
    Skip navigation links
    - - - - -
    -
    - - - -
    - -
    -
    - - -
    - - -
    - - - diff --git a/doc/bftsmart/tom/server/defaultservices/FileRecoverer.html b/doc/bftsmart/tom/server/defaultservices/FileRecoverer.html deleted file mode 100644 index c1f84c224..000000000 --- a/doc/bftsmart/tom/server/defaultservices/FileRecoverer.html +++ /dev/null @@ -1,419 +0,0 @@ - - - - - -FileRecoverer - - - - - - - - -
    - - -
    Skip navigation links
    - - - - -
    -
    - - - -
    - -
    -
    -
      -
    • Summary: 
      • -
    • Nested | 
      • -
    • Field | 
      • -
    • Constr | 
      • -
    • Method
      • -
    - -
    - - -
    - - -
    -
    bftsmart.tom.server.defaultservices
    -

    Class FileRecoverer

    -
    -
    -
      -
    • java.lang.Object
      • -
    • -
        -
      • bftsmart.tom.server.defaultservices.FileRecoverer
        • -
      -
      • -
    -
    -
      -
    • -
      -
      -
      public class FileRecoverer
-extends java.lang.Object
      -
      • -
    -
    -
    -
      -
    • - -
        -
      • - - -

        Constructor Summary

        - - - - - - - - -
        Constructors 
        Constructor and Description
        FileRecoverer(int replicaId, - java.lang.String defaultDir) 
        -
        • -
      - -
        -
      • - - -

        Method Summary

        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        All Methods Instance Methods Concrete Methods 
        Modifier and TypeMethod and Description
        intgetCkpLastConsensusId() 
        byte[]getCkpState(java.lang.String ckpPath) 
        byte[]getCkpStateHash() 
        java.lang.StringgetLatestFile(java.lang.String extention) 
        intgetLogLastConsensusId() 
        CommandsInfo[]getLogState(int index, - java.lang.String logPath) -
        Reads all log messages from the last log file created
        -
        CommandsInfo[]getLogState(long pointer, - int startOffset, - int number, - java.lang.String logPath) -
        Recover portions of the log for collaborative state transfer.
        -
        voidrecoverCkpHash(java.lang.String ckpPath) 
        voidtransferCkpState(java.nio.channels.SocketChannel sChannel, - java.lang.String ckpPath) 
        voidtransferLog(java.nio.channels.SocketChannel sChannel, - int index, - java.lang.String logPath) 
        -
          -
        • - - -

          Methods inherited from class java.lang.Object

          -clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
          • -
        -
        • -
      -
      • -
    -
    -
    -
      -
    • - -
        -
      • - - -

        Constructor Detail

        - - - -
          -
        • -

          FileRecoverer

          -
          public FileRecoverer(int replicaId,
-                     java.lang.String defaultDir)
          -
          • -
        -
        • -
      - -
        -
      • - - -

        Method Detail

        - - - -
          -
        • -

          getLogState

          -
          public CommandsInfo[] getLogState(int index,
-                                  java.lang.String logPath)
          -
          Reads all log messages from the last log file created
          -
          -
          Returns:
          -
          an array with batches of messages executed for each consensus
          -
          -
          • -
        - - - -
          -
        • -

          getLogState

          -
          public CommandsInfo[] getLogState(long pointer,
-                                  int startOffset,
-                                  int number,
-                                  java.lang.String logPath)
          -
          Recover portions of the log for collaborative state transfer.
          -
          -
          Parameters:
          -
          start - the index for which the commands start to be collected
          -
          number - the number of commands retrieved
          -
          Returns:
          -
          The commands for the period selected
          -
          -
          • -
        - - - -
          -
        • -

          getCkpState

          -
          public byte[] getCkpState(java.lang.String ckpPath)
          -
          • -
        - - - -
          -
        • -

          recoverCkpHash

          -
          public void recoverCkpHash(java.lang.String ckpPath)
          -
          • -
        - - - -
          -
        • -

          transferLog

          -
          public void transferLog(java.nio.channels.SocketChannel sChannel,
-                        int index,
-                        java.lang.String logPath)
          -
          • -
        - - - -
          -
        • -

          transferCkpState

          -
          public void transferCkpState(java.nio.channels.SocketChannel sChannel,
-                             java.lang.String ckpPath)
          -
          • -
        - - - -
          -
        • -

          getCkpStateHash

          -
          public byte[] getCkpStateHash()
          -
          • -
        - - - -
          -
        • -

          getCkpLastConsensusId

          -
          public int getCkpLastConsensusId()
          -
          • -
        - - - -
          -
        • -

          getLogLastConsensusId

          -
          public int getLogLastConsensusId()
          -
          • -
        - - - -
          -
        • -

          getLatestFile

          -
          public java.lang.String getLatestFile(java.lang.String extention)
          -
          • -
        -
        • -
      -
      • -
    -
    -
    - - -
    - - -
    Skip navigation links
    - - - - -
    -
    - - - -
    - -
    -
    -
      -
    • Summary: 
      • -
    • Nested | 
      • -
    • Field | 
      • -
    • Constr | 
      • -
    • Method
      • -
    - -
    - - -
    - - - diff --git a/doc/bftsmart/tom/server/defaultservices/StateLog.html b/doc/bftsmart/tom/server/defaultservices/StateLog.html deleted file mode 100644 index 9a01d6cc8..000000000 --- a/doc/bftsmart/tom/server/defaultservices/StateLog.html +++ /dev/null @@ -1,599 +0,0 @@ - - - - - -StateLog - - - - - - - - -
    - - -
    Skip navigation links
    - - - - -
    -
    - - - -
    - -
    -
    -
      -
    • Summary: 
      • -
    • Nested | 
      • -
    • Field | 
      • -
    • Constr | 
      • -
    • Method
      • -
    - -
    - - -
    - - -
    -
    bftsmart.tom.server.defaultservices
    -

    Class StateLog

    -
    -
    -
      -
    • java.lang.Object
      • -
    • -
        -
      • bftsmart.tom.server.defaultservices.StateLog
        • -
      -
      • -
    -
    -
      -
    • -
      -
      Direct Known Subclasses:
      -
      DiskStateLog
      -
      -
      -
      -
      public class StateLog
-extends java.lang.Object
      -
      This classes serves as a log for the state associated with the last checkpoint, and the message - batches received since the same checkpoint until the present. The state associated with the last - checkpoint together with all the batches of messages received so far, comprises this replica - current state
      -
      • -
    -
    -
    -
      -
    • - -
        -
      • - - -

        Constructor Summary

        - - - - - - - - - - - - - - -
        Constructors 
        Constructor and Description
        StateLog(int id, - byte[] initialState, - byte[] initialHash) 
        StateLog(int id, - int k) -
        Constructs a State log
        -
        StateLog(int id, - int k, - byte[] initialState, - byte[] initialHash) -
        Constructs a State log
        -
        -
        • -
      - -
        -
      • - - -

        Method Summary

        - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
        All Methods Instance Methods Concrete Methods 
        Modifier and TypeMethod and Description
        voidaddMessageBatch(byte[][] commands, - MessageContext[] msgCtx, - int lastConsensusId) -
        Adds a message batch to the log.
        -
        DefaultApplicationStategetApplicationState(int cid, - boolean setState) -
        Constructs a TransferableState using this log information
        -
        intgetLastCheckpointCID() -
        Retrieves the consensus ID for the last checkpoint
        -
        intgetLastCID() -
        Retrieves the consensus ID for the last messages batch delivered to the application
        -
        CommandsInfogetMessageBatch(int cid) -
        Returns a batch of messages, given its correspondent consensus ID
        -
        CommandsInfo[]getMessageBatches() -
        Retrieves all the stored batches kept since the last checkpoint
        -
        intgetNumBatches() -
        Retrieves the total number of stored batches kept since the last checkpoint
        -
        byte[]getState() -
        Retrieves the state associated with the last checkpoint
        -
        byte[]getStateHash() -
        Retrieves the hash of the state associated with the last checkpoint
        -
        voidnewCheckpoint(byte[] state, - byte[] stateHash, - int lastConsensusId) -
        Sets the state associated with the last checkpoint, and updates the consensus ID associated with it
        -
        voidsetLastCheckpointCID(int lastCheckpointCID) -
        Sets the consensus ID for the last checkpoint
        -
        voidsetLastCID(int lastCID) -
        Sets the consensus ID for the last messages batch delivered to the application
        -
        voidupdate(DefaultApplicationState transState) -
        Updates this log, according to the information contained in the TransferableState object
        -
        -
          -
        • - - -

          Methods inherited from class java.lang.Object

          -clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
          • -
        -
        • -
      -
      • -
    -
    -
    -
      -
    • - -
        -
      • - - -

        Constructor Detail

        - - - -
          -
        • -

          StateLog

          -
          public StateLog(int id,
-                int k,
-                byte[] initialState,
-                byte[] initialHash)
          -
          Constructs a State log
          -
          -
          Parameters:
          -
          id -
          -
          k - The checkpoint period
          -
          initialState -
          -
          initialHash -
          -
          -
          • -
        - - - -
          -
        • -

          StateLog

          -
          public StateLog(int id,
-                int k)
          -
          Constructs a State log
          -
          -
          Parameters:
          -
          id -
          -
          k - The checkpoint period
          -
          -
          • -
        - - - -
          -
        • -

          StateLog

          -
          public StateLog(int id,
-                byte[] initialState,
-                byte[] initialHash)
          -
          • -
        -
        • -
      - -
        -
      • - - -

        Method Detail

        - - - -
          -
        • -

          newCheckpoint

          -
          public void newCheckpoint(byte[] state,
-                          byte[] stateHash,
-                          int lastConsensusId)
          -
          Sets the state associated with the last checkpoint, and updates the consensus ID associated with it
          -
          -
          Parameters:
          -
          state - State associated with the last checkpoint
          -
          stateHash -
          -
          lastConsensusId -
          -
          -
          • -
        - - - -
          -
        • -

          setLastCheckpointCID

          -
          public void setLastCheckpointCID(int lastCheckpointCID)
          -
          Sets the consensus ID for the last checkpoint
          -
          -
          Parameters:
          -
          lastCheckpointCID - Consensus ID for the last checkpoint
          -
          -
          • -
        - - - -
          -
        • -

          getLastCheckpointCID

          -
          public int getLastCheckpointCID()
          -
          Retrieves the consensus ID for the last checkpoint
          -
          -
          Returns:
          -
          Consensus ID for the last checkpoint, or -1 if none was obtained
          -
          -
          • -
        - - - -
          -
        • -

          setLastCID

          -
          public void setLastCID(int lastCID)
          -
          Sets the consensus ID for the last messages batch delivered to the application
          -
          -
          Parameters:
          -
          lastCID - the consensus ID for the last messages batch delivered to the application
          -
          -
          • -
        - - - -
          -
        • -

          getLastCID

          -
          public int getLastCID()
          -
          Retrieves the consensus ID for the last messages batch delivered to the application
          -
          -
          Returns:
          -
          Consensus ID for the last messages batch delivered to the application
          -
          -
          • -
        - - - -
          -
        • -

          getState

          -
          public byte[] getState()
          -
          Retrieves the state associated with the last checkpoint
          -
          -
          Returns:
          -
          State associated with the last checkpoint
          -
          -
          • -
        - - - -
          -
        • -

          getStateHash

          -
          public byte[] getStateHash()
          -
          Retrieves the hash of the state associated with the last checkpoint
          -
          -
          Returns:
          -
          Hash of the state associated with the last checkpoint
          -
          -
          • -
        - - - -
          -
        • -

          addMessageBatch

          -
          public void addMessageBatch(byte[][] commands,
-                            MessageContext[] msgCtx,
-                            int lastConsensusId)
          -
          Adds a message batch to the log. This batches should be added to the log - in the same order in which they are delivered to the application. Only - the 'k' batches received after the last checkpoint are supposed to be kept
          -
          -
          Parameters:
          -
          commands - The batch of messages to be kept.
          -
          msgCtx - The message contexts related to the commands
          -
          lastConsensusId -
          -
          -
          • -
        - - - -
          -
        • -

          getMessageBatch

          -
          public CommandsInfo getMessageBatch(int cid)
          -
          Returns a batch of messages, given its correspondent consensus ID
          -
          -
          Parameters:
          -
          cid - Consensus ID associated with the batch to be fetched
          -
          Returns:
          -
          The batch of messages associated with the batch correspondent consensus ID
          -
          -
          • -
        - - - -
          -
        • -

          getMessageBatches

          -
          public CommandsInfo[] getMessageBatches()
          -
          Retrieves all the stored batches kept since the last checkpoint
          -
          -
          Returns:
          -
          All the stored batches kept since the last checkpoint
          -
          -
          • -
        - - - -
          -
        • -

          getNumBatches

          -
          public int getNumBatches()
          -
          Retrieves the total number of stored batches kept since the last checkpoint
          -
          -
          Returns:
          -
          The total number of stored batches kept since the last checkpoint
          -
          -
          • -
        - - - -
          -
        • -

          getApplicationState

          -
          public DefaultApplicationState getApplicationState(int cid,
-                                                   boolean setState)
          -
          Constructs a TransferableState using this log information
          -
          -
          Parameters:
          -
          cid - Consensus ID correspondent to desired state
          -
          setState -
          -
          Returns:
          -
          TransferableState Object containing this log information
          -
          -
          • -
        - - - -
          -
        • -

          update

          -
          public void update(DefaultApplicationState transState)
          -
          Updates this log, according to the information contained in the TransferableState object
          -
          -
          Parameters:
          -
          transState - TransferableState object containing the information which is used to updated this log
          -
          -
          • -
        -
        • -
      -
      • -
    -
    -
    - - -
    - - -
    Skip navigation links
    - - - - -
    -
    - - - -
    - -
    -
    -
      -
    • Summary: 
      • -
    • Nested | 
      • -
    • Field | 
      • -
    • Constr | 
      • -
    • Method
      • -
    - -
    - - -
    - - - diff --git a/doc/bftsmart/tom/server/defaultservices/durability/DurabilityCoordinator.html b/doc/bftsmart/tom/server/defaultservices/durability/DurabilityCoordinator.html index da0071839..60fce60fd 100644 --- a/doc/bftsmart/tom/server/defaultservices/durability/DurabilityCoordinator.html +++ b/doc/bftsmart/tom/server/defaultservices/durability/DurabilityCoordinator.html @@ -2,9 +2,9 @@ - + DurabilityCoordinator - + @@ -161,12 +161,16 @@

    Method Summary

    abstract byte[][] appExecuteBatch(byte[][] commands, - MessageContext[] msgCtxs)  + MessageContext[] msgCtxs)     +
    Execute a batch of ordered requests
    + abstract byte[] appExecuteUnordered(byte[] command, - MessageContext msgCtx)  + MessageContext msgCtx) +
    Execute an unordered request
    + byte[][] @@ -188,7 +192,9 @@

    Method Summary

    abstract byte[] -getSnapshot()  +getSnapshot() +
    Returns a serialized snapshot of the application state
    + bftsmart.statemanagement.strategy.durability.CSTState @@ -211,12 +217,15 @@

    Method Summary

    abstract void -installSnapshot(byte[] state)  +installSnapshot(byte[] state) +
    Given a snapshot received from the state transfer protocol, install it
    + void -noOp(int CID, - MessageContext msgCtx) +noOp(int CID, + byte[][] operations, + MessageContext[] msgCtxs)
    This method is invoked by ServiceReplica to indicate that a consensus instance finished without delivering anything to the application (e.g., an instance only decided a single reconfiguration operation.
    @@ -233,7 +242,9 @@

    Method Summary

    void -setReplicaContext(ReplicaContext replicaContext)  +setReplicaContext(ReplicaContext replicaContext) +
    Sets the replica context
    + int @@ -293,6 +304,9 @@

    executeBatch

    Specified by:
    executeBatch in interface BatchExecutable
    +
    Parameters:
    +
    commands - The batch of requests
    +
    msgCtxs - The context associated to each request
    Returns:
    @@ -353,9 +367,13 @@

    getState

  • setReplicaContext

    public void setReplicaContext(ReplicaContext replicaContext)
    +
    Description copied from interface: Recoverable
    +
    Sets the replica context
    Specified by:
    setReplicaContext in interface Recoverable
    +
    Parameters:
    +
    replicaContext - The replica context
    • @@ -439,15 +457,16 @@

    Op

    - +

    • noOp

      public void noOp(int CID,
-                 MessageContext msgCtx)
      -
      Description copied from interface: Recoverable
      + byte[][] operations, + MessageContext[] msgCtxs) +
      Description copied from interface: Recoverable
      This method is invoked by ServiceReplica to indicate that a consensus instance finished without delivering anything to the application (e.g., an instance only decided a single reconfiguration operation. or an instance where the client @@ -457,10 +476,11 @@

      noOp

      logs used within the state transfer, but never deliver it to the application
      Specified by:
      -
      noOp in interface Recoverable
      +
      noOp in interface Recoverable
      Parameters:
      CID - the consensus instance where the aforementioned condition occurred
      -
      msgCtx - Message context associated with the consensus instance. furthermore +
      operations - Operations decided in CID
      +
      msgCtxs - Message context associated with the consensus instance. furthermore msgCtx.getConsensusId() will be equal to CID.
      • @@ -472,6 +492,11 @@

      noOp

    • installSnapshot

      public abstract void installSnapshot(byte[] state)
      +
      Given a snapshot received from the state transfer protocol, install it
      +
      +
      Parameters:
      +
      state - The serialized snapshot
      +
    @@ -481,6 +506,11 @@

    installSnapshot

  • getSnapshot

    public abstract byte[] getSnapshot()
    +
    Returns a serialized snapshot of the application state
    +
    +
    Returns:
    +
    A serialized snapshot of the application state
    +
    •  @@ -491,6 +521,14 @@

    getSnapshot

    appExecuteBatch

    public abstract byte[][] appExecuteBatch(byte[][] commands,
                                          MessageContext[] msgCtxs)
    +
    Execute a batch of ordered requests
    +
    +
    Parameters:
    +
    commands - The batch of requests
    +
    msgCtxs - The context associated to each request
    +
    Returns:
    +
    the respective replies for each request
    +
    @@ -501,6 +539,14 @@

    appExecuteBatch

    appExecuteUnordered

    public abstract byte[] appExecuteUnordered(byte[] command,
                                            MessageContext msgCtx)
    +
    Execute an unordered request
    +
    +
    Parameters:
    +
    command - The unordered request
    +
    msgCtx - The context associated to the request
    +
    Returns:
    +
    the reply for the request issued by the client
    +
    diff --git a/doc/bftsmart/tom/server/defaultservices/durability/package-frame.html b/doc/bftsmart/tom/server/defaultservices/durability/package-frame.html index 705fb76e2..5117c4c0f 100644 --- a/doc/bftsmart/tom/server/defaultservices/durability/package-frame.html +++ b/doc/bftsmart/tom/server/defaultservices/durability/package-frame.html @@ -2,9 +2,9 @@ - + bftsmart.tom.server.defaultservices.durability - + diff --git a/doc/bftsmart/tom/server/defaultservices/durability/package-summary.html b/doc/bftsmart/tom/server/defaultservices/durability/package-summary.html index f23189ca5..55c49f8fc 100644 --- a/doc/bftsmart/tom/server/defaultservices/durability/package-summary.html +++ b/doc/bftsmart/tom/server/defaultservices/durability/package-summary.html @@ -2,9 +2,9 @@ - + bftsmart.tom.server.defaultservices.durability - + diff --git a/doc/bftsmart/tom/server/defaultservices/durability/package-tree.html b/doc/bftsmart/tom/server/defaultservices/durability/package-tree.html index 594c060f5..d3249df08 100644 --- a/doc/bftsmart/tom/server/defaultservices/durability/package-tree.html +++ b/doc/bftsmart/tom/server/defaultservices/durability/package-tree.html @@ -2,9 +2,9 @@ - + bftsmart.tom.server.defaultservices.durability Class Hierarchy - + diff --git a/doc/bftsmart/tom/server/defaultservices/package-frame.html b/doc/bftsmart/tom/server/defaultservices/package-frame.html index 8ae716eb4..488f3e054 100644 --- a/doc/bftsmart/tom/server/defaultservices/package-frame.html +++ b/doc/bftsmart/tom/server/defaultservices/package-frame.html @@ -2,9 +2,9 @@ - + bftsmart.tom.server.defaultservices - + diff --git a/doc/bftsmart/tom/server/defaultservices/package-summary.html b/doc/bftsmart/tom/server/defaultservices/package-summary.html index f8400c564..39f87db1d 100644 --- a/doc/bftsmart/tom/server/defaultservices/package-summary.html +++ b/doc/bftsmart/tom/server/defaultservices/package-summary.html @@ -2,9 +2,9 @@ - + bftsmart.tom.server.defaultservices - + diff --git a/doc/bftsmart/tom/server/defaultservices/package-tree.html b/doc/bftsmart/tom/server/defaultservices/package-tree.html index 25b251eb0..4708f76bf 100644 --- a/doc/bftsmart/tom/server/defaultservices/package-tree.html +++ b/doc/bftsmart/tom/server/defaultservices/package-tree.html @@ -2,9 +2,9 @@ - + bftsmart.tom.server.defaultservices Class Hierarchy - + diff --git a/doc/bftsmart/tom/server/package-frame.html b/doc/bftsmart/tom/server/package-frame.html index d894fe4bc..bfa2c020c 100644 --- a/doc/bftsmart/tom/server/package-frame.html +++ b/doc/bftsmart/tom/server/package-frame.html @@ -2,9 +2,9 @@ - + bftsmart.tom.server - + @@ -15,7 +15,6 @@

    Interfaces

    • BatchExecutable
    • Executable
      • -
    • FIFOExecutable
    • Recoverable
    • Replier
    • RequestVerifier
      • diff --git a/doc/bftsmart/tom/server/package-summary.html b/doc/bftsmart/tom/server/package-summary.html index 67fbd56d8..ec1ab7bfd 100644 --- a/doc/bftsmart/tom/server/package-summary.html +++ b/doc/bftsmart/tom/server/package-summary.html @@ -2,9 +2,9 @@ - + bftsmart.tom.server - + @@ -95,26 +95,19 @@

      Package bftsmart.tom.server

      -FIFOExecutable - -
      Executables that implement this interface will get requests - delivered in FIFO order.
      - - - Recoverable
      Classes that implement this interface should implement a state transfer protocol.
      - + Replier -
      Objects that implement this interface can be passed to the ServiceReplica - and manage replies in a custom manner.
      +
      Provides support for building custom reply management + to be used in the ServiceReplica.
      - + RequestVerifier
      Classes that implement this interface are invoked within @@ -122,7 +115,7 @@

      Package bftsmart.tom.server

      in order to enforce the "external validity".
      - + SingleExecutable
      Executables that implement this interface will receive client requests individually.
      diff --git a/doc/bftsmart/tom/server/package-tree.html b/doc/bftsmart/tom/server/package-tree.html index c392560fd..160da379f 100644 --- a/doc/bftsmart/tom/server/package-tree.html +++ b/doc/bftsmart/tom/server/package-tree.html @@ -2,9 +2,9 @@ - + bftsmart.tom.server Class Hierarchy - + @@ -81,11 +81,7 @@

      Interface Hierarchy

    • bftsmart.tom.server.Executable
    • bftsmart.tom.server.Recoverable
      • diff --git a/doc/bftsmart/tom/util/Extractor.html b/doc/bftsmart/tom/util/Extractor.html index e1a60da1f..c80b88b0a 100644 --- a/doc/bftsmart/tom/util/Extractor.html +++ b/doc/bftsmart/tom/util/Extractor.html @@ -2,9 +2,9 @@ - + Extractor - + @@ -49,7 +49,7 @@
      • Frames
        • @@ -125,7 +125,9 @@

        Method Summary

        bftsmart.tom.core.messages.TOMMessage extractResponse(bftsmart.tom.core.messages.TOMMessage[] replies, int sameContent, - int lastReceived)  + int lastReceived) +
        Extracts a reply given a set of replies from a set of replicas.
        + @@ -151,6 +153,14 @@

        extractResponse

        bftsmart.tom.core.messages.TOMMessage extractResponse(bftsmart.tom.core.messages.TOMMessage[] replies,
                                                       int sameContent,
                                                       int lastReceived)
        +
        Extracts a reply given a set of replies from a set of replicas.
        +
        +
        Parameters:
        +
        replies - Set of replies from a set of replicas.
        +
        sameContent - Whether or not the replies are supposed to have the same content
        +
        lastReceived - Last reply received from the replicas. This is an index in relation to the `replies` parameter.
        +
        Returns:
        +
      @@ -181,7 +191,7 @@

      extractResponse

      • Frames
        • diff --git a/doc/bftsmart/tom/util/KeyLoader.html b/doc/bftsmart/tom/util/KeyLoader.html new file mode 100644 index 000000000..66e88cb0c --- /dev/null +++ b/doc/bftsmart/tom/util/KeyLoader.html @@ -0,0 +1,316 @@ + + + + + +KeyLoader + + + + + + + + +
        + + +
        Skip navigation links
        + + + + +
        +
        + + + +
        + +
        +
        +
          +
        • Summary: 
          • +
        • Nested | 
          • +
        • Field | 
          • +
        • Constr | 
          • +
        • Method
          • +
        +
          +
        • Detail: 
          • +
        • Field | 
          • +
        • Constr | 
          • +
        • Method
          • +
        +
        + + +
        + + +
        +
        bftsmart.tom.util
        +

        Interface KeyLoader

        +
        +
        +
        +
          +
        • +
          +
          +
          public interface KeyLoader
          +
          The KeyLoader interface is used internally by BFT-SMaRt to load signature keys from disk. Developers can use implementations of this + interface so that if their application uses key structures similar to the library, these keys can be shared by both the library and + the application to simplify key management.
          +
          • +
        +
        +
        +
          +
        • + +
            +
          • + + +

            Method Summary

            + + + + + + + + + + + + + + + + + + + + + + +
            All Methods Instance Methods Abstract Methods 
            Modifier and TypeMethod and Description
            java.lang.StringgetSignatureAlgorithm() +
            Get the signature algorithm specified by the key.
            +
            java.security.PrivateKeyloadPrivateKey() +
            Fetches the private key for this replica/client.
            +
            java.security.PublicKeyloadPublicKey() +
            Fetches the public key for this replica/client.
            +
            java.security.PublicKeyloadPublicKey(int id) +
            Fetches the public key for the specified replica/client.
            +
            +
            • +
          +
          • +
        +
        +
        +
          +
        • + +
            +
          • + + +

            Method Detail

            + + + +
              +
            • +

              loadPublicKey

              +
              java.security.PublicKey loadPublicKey(int id)
+                               throws java.io.IOException,
+                                      java.security.NoSuchAlgorithmException,
+                                      java.security.spec.InvalidKeySpecException,
+                                      java.security.cert.CertificateException
              +
              Fetches the public key for the specified replica/client.
              +
              +
              Parameters:
              +
              id - ID of the respective replica/client.
              +
              Returns:
              +
              The replica/client's public key.
              +
              Throws:
              +
              java.io.IOException - If there was an error reading the key file.
              +
              java.security.NoSuchAlgorithmException - If the algorithm specified in the configuration does not exist.
              +
              java.security.spec.InvalidKeySpecException - If the key specs are not properly set.
              +
              java.security.cert.CertificateException - If there was an error loading the public key certificate.
              +
              +
              • +
            + + + +
              +
            • +

              loadPublicKey

              +
              java.security.PublicKey loadPublicKey()
+                               throws java.io.IOException,
+                                      java.security.NoSuchAlgorithmException,
+                                      java.security.spec.InvalidKeySpecException,
+                                      java.security.cert.CertificateException
              +
              Fetches the public key for this replica/client.
              +
              +
              Returns:
              +
              The replica/client's public key.
              +
              Throws:
              +
              java.io.IOException - If there was an error reading the key file.
              +
              java.security.NoSuchAlgorithmException - If the algorithm specified by the key does not exist.
              +
              java.security.spec.InvalidKeySpecException - If the key specs are not properly set.
              +
              java.security.cert.CertificateException - If there was an error loading the public key certificate.
              +
              +
              • +
            + + + +
              +
            • +

              loadPrivateKey

              +
              java.security.PrivateKey loadPrivateKey()
+                                 throws java.io.IOException,
+                                        java.security.NoSuchAlgorithmException,
+                                        java.security.spec.InvalidKeySpecException
              +
              Fetches the private key for this replica/client.
              +
              +
              Returns:
              +
              The replica/client's public key.
              +
              Throws:
              +
              java.io.IOException - If there was an error reading the key file.
              +
              java.security.NoSuchAlgorithmException - If the algorithm specified in the configuration does not exist.
              +
              java.security.spec.InvalidKeySpecException - If the key specs are not properly set .
              +
              +
              • +
            + + + +
              +
            • +

              getSignatureAlgorithm

              +
              java.lang.String getSignatureAlgorithm()
              +
              Get the signature algorithm specified by the key.
              +
              +
              Returns:
              +
              The signature algorithm specified by the key.
              +
              +
              • +
            +
            • +
          +
          • +
        +
        +
        + + +
        + + +
        Skip navigation links
        + + + + +
        +
        + + + +
        + +
        +
        +
          +
        • Summary: 
          • +
        • Nested | 
          • +
        • Field | 
          • +
        • Constr | 
          • +
        • Method
          • +
        +
          +
        • Detail: 
          • +
        • Field | 
          • +
        • Constr | 
          • +
        • Method
          • +
        +
        + + +
        + + + diff --git a/doc/bftsmart/tom/util/RSAKeyPairGenerator.html b/doc/bftsmart/tom/util/RSAKeyPairGenerator.html deleted file mode 100644 index 9694bd167..000000000 --- a/doc/bftsmart/tom/util/RSAKeyPairGenerator.html +++ /dev/null @@ -1,301 +0,0 @@ - - - - - -RSAKeyPairGenerator - - - - - - - - -
        - - -
        Skip navigation links
        - - - - -
        -
        - - - -
        - -
        -
        -
          -
        • Summary: 
          • -
        • Nested | 
          • -
        • Field | 
          • -
        • Constr | 
          • -
        • Method
          • -
        - -
        - - -
        - - -
        -
        bftsmart.tom.util
        -

        Class RSAKeyPairGenerator

        -
        -
        -
          -
        • java.lang.Object
          • -
        • -
            -
          • bftsmart.tom.util.RSAKeyPairGenerator
            • -
          -
          • -
        -
        -
          -
        • -
          -
          -
          public class RSAKeyPairGenerator
-extends java.lang.Object
          -
          Utility class used to generate a key pair for some process id on - config/keys/publickey and config/keys/privatekey
          -
          • -
        -
        -
        -
          -
        • - -
            -
          • - - -

            Constructor Summary

            - - - - - - - - -
            Constructors 
            Constructor and Description
            RSAKeyPairGenerator() -
            Creates a new instance of KeyPairGenerator
            -
            -
            • -
          - -
            -
          • - - -

            Method Summary

            - - - - - - - - - - - - - - -
            All Methods Static Methods Instance Methods Concrete Methods 
            Modifier and TypeMethod and Description
            static voidmain(java.lang.String[] args) 
            voidrun(int id, - int size) -
            Generate the key pair for the process with id = and put it on the - files config/keys/publickey and config/keys/privatekey
            -
            -
              -
            • - - -

              Methods inherited from class java.lang.Object

              -clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
              • -
            -
            • -
          -
          • -
        -
        -
        -
          -
        • - -
            -
          • - - -

            Constructor Detail

            - - - -
              -
            • -

              RSAKeyPairGenerator

              -
              public RSAKeyPairGenerator()
              -
              Creates a new instance of KeyPairGenerator
              -
              • -
            -
            • -
          - -
            -
          • - - -

            Method Detail

            - - - -
              -
            • -

              run

              -
              public void run(int id,
-                int size)
-         throws java.lang.Exception
              -
              Generate the key pair for the process with id = and put it on the - files config/keys/publickey and config/keys/privatekey
              -
              -
              Parameters:
              -
              id - the id of the process to generate key
              -
              Throws:
              -
              java.lang.Exception - something goes wrong when writing the files
              -
              -
              • -
            - - - -
              -
            • -

              main

              -
              public static void main(java.lang.String[] args)
              -
              • -
            -
            • -
          -
          • -
        -
        -
        - - -
        - - -
        Skip navigation links
        - - - - -
        -
        - - - -
        - -
        -
        -
          -
        • Summary: 
          • -
        • Nested | 
          • -
        • Field | 
          • -
        • Constr | 
          • -
        • Method
          • -
        - -
        - - -
        - - - diff --git a/doc/bftsmart/tom/util/package-frame.html b/doc/bftsmart/tom/util/package-frame.html index 1804aa02a..5cbe2a023 100644 --- a/doc/bftsmart/tom/util/package-frame.html +++ b/doc/bftsmart/tom/util/package-frame.html @@ -2,9 +2,9 @@ - + bftsmart.tom.util - + @@ -14,10 +14,7 @@

        Interfaces

        -

        Classes

        -
      diff --git a/doc/bftsmart/tom/util/package-summary.html b/doc/bftsmart/tom/util/package-summary.html index 58878a24d..6b8ef3d65 100644 --- a/doc/bftsmart/tom/util/package-summary.html +++ b/doc/bftsmart/tom/util/package-summary.html @@ -2,9 +2,9 @@ - + bftsmart.tom.util - + @@ -88,22 +88,10 @@

      Package bftsmart.tom.util

      ServiceProxy.
      - - - -
    • - - - - - - - - - + + diff --git a/doc/bftsmart/tom/util/package-tree.html b/doc/bftsmart/tom/util/package-tree.html index 491be7e7e..6babcd012 100644 --- a/doc/bftsmart/tom/util/package-tree.html +++ b/doc/bftsmart/tom/util/package-tree.html @@ -2,9 +2,9 @@ - +bftsmart.tom.util Class Hierarchy - + @@ -76,17 +76,10 @@

      Hierarchy For Package bftsmart.tom.util

      -

      Class Hierarchy

      -

      Interface Hierarchy

      diff --git a/doc/constant-values.html b/doc/constant-values.html index aa17a177d..0b28e3e1a 100644 --- a/doc/constant-values.html +++ b/doc/constant-values.html @@ -2,9 +2,9 @@ - +Constant Field Values - + diff --git a/doc/deprecated-list.html b/doc/deprecated-list.html index ada164ea9..90aaed1a9 100644 --- a/doc/deprecated-list.html +++ b/doc/deprecated-list.html @@ -2,9 +2,9 @@ - +Deprecated List - + @@ -71,6 +71,40 @@

      Deprecated API

      Contents

      + +
      +
      + + +
        +
      • +
      Class Summary 
      ClassDescription
      RSAKeyPairGenerator
      KeyLoader -
      Utility class used to generate a key pair for some process id on - config/keys/publickey and config/keys/privatekey
      +
      The KeyLoader interface is used internally by BFT-SMaRt to load signature keys from disk.
      + + + + + + + + + + + + + + + + + + + + + +
      Deprecated Methods 
      Method and Description
      bftsmart.tom.ServiceReplica.joinMsgReceived(VMMessage) 
      bftsmart.tom.ServiceReplica.receiveMessages(int[], int[], int[], CertifiedDecision[], TOMMessage[][]) 
      bftsmart.tom.ServiceReplica.receiveReadonlyMessage(TOMMessage, MessageContext) 
      bftsmart.tom.ServiceProxy.reconfigureTo(View) 
      bftsmart.tom.MessageContext.setLastInBatch() 
      +
      • +
    diff --git a/doc/help-doc.html b/doc/help-doc.html index d0c96519f..5baab1457 100644 --- a/doc/help-doc.html +++ b/doc/help-doc.html @@ -2,9 +2,9 @@ - + API Help - + diff --git a/doc/index-all.html b/doc/index-all.html index d359ed729..f6187f191 100644 --- a/doc/index-all.html +++ b/doc/index-all.html @@ -2,9 +2,9 @@ - + Index - + @@ -68,34 +68,60 @@
    -
    A B C D E F G I J L M N O R S T  +
    A B C D E G I J K L M N O R S V 

    A

    -
    appExecuteBatch(byte[][], MessageContext[]) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
    -
     
    +
    addServer(int, String, int) - Method in class bftsmart.reconfiguration.VMServices
    +
    +
    Adds a new server to the group
    +
    +
    appExecuteBatch(byte[][], MessageContext[], boolean) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
    +
    +
    Execute a batch of ordered requests
    +
    appExecuteBatch(byte[][], MessageContext[]) - Method in class bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator
    -
     
    +
    +
    Execute a batch of ordered requests
    +
    appExecuteOrdered(byte[], MessageContext) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    -
     
    +
    +
    Execute a batch of ordered requests
    +
    appExecuteUnordered(byte[], MessageContext) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
    -
     
    +
    +
    Execute an unordered request
    +
    appExecuteUnordered(byte[], MessageContext) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    -
     
    +
    +
    Execute an unordered request
    +
    appExecuteUnordered(byte[], MessageContext) - Method in class bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator
    -
     
    +
    +
    Execute an unordered request
    +
    AsynchServiceProxy - Class in bftsmart.tom
    This class is an extension of 'ServiceProxy' that can waits for replies asynchronously.
    AsynchServiceProxy(int) - Constructor for class bftsmart.tom.AsynchServiceProxy
    -
     
    +
    +
    Constructor
    +
    AsynchServiceProxy(int, String) - Constructor for class bftsmart.tom.AsynchServiceProxy
    -
     
    -
    AsynchServiceProxy(int, String, Comparator<byte[]>, Extractor) - Constructor for class bftsmart.tom.AsynchServiceProxy
    -
     
    +
    +
    Constructor
    +
    +
    AsynchServiceProxy(int, String, KeyLoader) - Constructor for class bftsmart.tom.AsynchServiceProxy
    +
    +
    Constructor
    +
    +
    AsynchServiceProxy(int, String, Comparator<byte[]>, Extractor, KeyLoader) - Constructor for class bftsmart.tom.AsynchServiceProxy
    +
    +
    Constructor
    +
    @@ -122,13 +148,9 @@

    C

    canSendLock - Variable in class bftsmart.tom.ServiceProxy
     
    cleanAsynchRequest(int) - Method in class bftsmart.tom.AsynchServiceProxy
    -
     
    -
    close() - Method in class bftsmart.tom.TOMSender
    -
     
    -
    computeHash(byte[]) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
    -
     
    -
    computeHash(byte[]) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    -
     
    +
    +
    Purges all information associated to the request.
    +
    @@ -141,7 +163,9 @@

    D

    'BatchExecutable'.
    DefaultRecoverable() - Constructor for class bftsmart.tom.server.defaultservices.DefaultRecoverable
    -
     
    +
    +
    Constructor
    +
    DefaultSingleRecoverable - Class in bftsmart.tom.server.defaultservices
    This class provides a basic state transfer protocol using the interface @@ -179,8 +203,6 @@

    E

    Method called to execute a request totally ordered.
    -
    executeOrderedFIFO(byte[], MessageContext, int, int) - Method in interface bftsmart.tom.server.FIFOExecutable
    -
     
    executeUnordered(byte[], MessageContext) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
     
    executeUnordered(byte[], MessageContext) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    @@ -191,25 +213,14 @@

    E

    Method called to execute a request totally ordered.
    -
    executeUnorderedFIFO(byte[], MessageContext, int, int) - Method in interface bftsmart.tom.server.FIFOExecutable
    -
     
    Extractor - Interface in bftsmart.tom.util
    Provides support for building custom response extractors to be used in the ServiceProxy.
    extractResponse(TOMMessage[], int, int) - Method in interface bftsmart.tom.util.Extractor
    -
     
    - - - - -

    F

    -
    -
    FIFOExecutable - Interface in bftsmart.tom.server
    -
    Executables that implement this interface will get requests - delivered in FIFO order.
    +
    Extracts a reply given a set of replies from a set of replicas.
    @@ -217,14 +228,11 @@

    F

    G

    -
    generateOperationId() - Method in class bftsmart.tom.TOMSender
    -
     
    -
    generateRequestId(TOMMessageType) - Method in class bftsmart.tom.TOMSender
    -
     
    -
    getCommunicationSystem() - Method in class bftsmart.tom.TOMSender
    -
     
    getConsensusId() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the ID of the consensus in which this request was ordered (-1 + if readonly).
    +
    getCurrentStateHash() - Method in class bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator
     
    getCurrentView() - Method in class bftsmart.tom.ReplicaContext
    @@ -232,69 +240,129 @@

    G

    Returns the current view of the replica group.
    getFirstInBatch() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the first request in the ordered batch
    +
    +
    getId() - Method in class bftsmart.tom.ServiceReplica
    +
    +
    Replica ID
    +
    getInvokeTimeout() - Method in class bftsmart.tom.ServiceProxy
    Get the amount of time (in seconds) that this proxy will wait for servers replies before returning null.
    getInvokeUnorderedHashedTimeout() - Method in class bftsmart.tom.ServiceProxy
    -
     
    +
    +
    Get the amount of time (in seconds) that this proxy will wait for + servers unordered hashed replies before returning null.
    +
    getLeader() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the id of the leader replica in which this message was ordered + (-1 if readonly).
    +
    getNonces() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the nonces as generated by the leader
    +
    getNumOfNonces() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the number of nonces as generated by the leader
    +
    getOperationId() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the operation ID
    +
    getOperationId() - Method in class bftsmart.tom.RequestContext
    -
     
    -
    getProcessId() - Method in class bftsmart.tom.TOMSender
    -
     
    +
    +
    Returns the operation ID
    +
    getProof() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the proof for the consensus.
    +
    getRegency() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the current regency in which the message was ordered.
    +
    getReplicaContext() - Method in class bftsmart.tom.ServiceReplica
    Obtains the current replica context (getting access to several information and capabilities of the replication engine).
    getReplyListener() - Method in class bftsmart.tom.RequestContext
    -
     
    +
    +
    Returns the reply listener associated to the request
    +
    +
    getReplyQuorum() - Method in class bftsmart.tom.ServiceProxy
    +
    +
    Retrieves the required quorum size for the amount of replies
    +
    getReplyServer() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the ID of the server expected to send a non-hashed reply when the clients sends an unordered hashed request.
    +
    getReqId() - Method in class bftsmart.tom.RequestContext
    -
     
    +
    +
    Returns the request unique ID
    +
    +
    getRequest() - Method in class bftsmart.tom.RequestContext
    +
    +
    The payload of the request
    +
    getRequestType() - Method in class bftsmart.tom.RequestContext
    -
     
    +
    +
    Returns the request type
    +
    getSeed() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the seed as generated by the leader
    +
    getSender() - Method in class bftsmart.tom.MessageContext
    -
    Returns the sender of the message
    +
    Returns the ID of the client that sent the message
    getSendingTime() - Method in class bftsmart.tom.RequestContext
    -
     
    +
    +
    Returns the sending time
    +
    getSequence() - Method in class bftsmart.tom.MessageContext
    -
     
    -
    getSerialVersionUID() - Static method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the request's sequence number
    +
    getServerCommunicationSystem() - Method in class bftsmart.tom.ReplicaContext
    -
     
    +
    +
    Returns the replica's communication system
    +
    +
    getServerCommunicationSystem() - Method in class bftsmart.tom.ServiceReplica
    +
    +
    Obtains the current replica communication system.
    +
    getSession() - Method in class bftsmart.tom.MessageContext
    -
     
    -
    getSession() - Method in class bftsmart.tom.TOMSender
    -
     
    +
    +
    Returns the client's session
    +
    getSignature() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the client's signature.
    +
    +
    getSignatureAlgorithm() - Method in interface bftsmart.tom.util.KeyLoader
    +
    +
    Get the signature algorithm specified by the key.
    +
    getSnapshot() - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
    -
     
    +
    +
    Returns a serialized snapshot of the application state
    +
    getSnapshot() - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    -
     
    +
    +
    Returns a serialized snapshot of the application state
    +
    getSnapshot() - Method in class bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator
    -
     
    +
    +
    Returns a serialized snapshot of the application state
    +
    getState(int, boolean) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
     
    getState(int, boolean) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    @@ -328,36 +396,39 @@

    G

    Returns the controller of the replica's view
    getTargets() - Method in class bftsmart.tom.RequestContext
    -
     
    +
    +
    Returns the IDs of the targets to which the request was sent
    +
    getTimestamp() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the timestamp as generated by the leader
    +
    getType() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the request type
    +
    getViewID() - Method in class bftsmart.tom.MessageContext
    -
     
    -
    getViewManager() - Method in class bftsmart.tom.TOMSender
    -
     
    +
    +
    Returns the view ID
    +

    I

    -
    init(int) - Method in class bftsmart.tom.TOMSender
    +
    installSnapshot(byte[]) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
    -
    This method initializes the object - TODO: Ask if this method cannot be protected (compiles, but....)
    +
    Given a snapshot received from the state transfer protocol, install it
    -
    init(int, String) - Method in class bftsmart.tom.TOMSender
    -
     
    -
    initLog() - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    -
     
    -
    installSnapshot(byte[]) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
    -
     
    installSnapshot(byte[]) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    -
     
    +
    +
    Given a snapshot received from the state transfer protocol, install it
    +
    installSnapshot(byte[]) - Method in class bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator
    -
     
    +
    +
    Given a snapshot received from the state transfer protocol, install it
    +
    invoke(byte[], TOMMessageType) - Method in class bftsmart.tom.ServiceProxy
    This method sends a request to the replicas, and returns the related reply.
    @@ -365,19 +436,33 @@

    I

    invokeAsynchRequest(byte[], ReplyListener, TOMMessageType) - Method in class bftsmart.tom.AsynchServiceProxy
     
    invokeAsynchRequest(byte[], int[], ReplyListener, TOMMessageType) - Method in class bftsmart.tom.AsynchServiceProxy
    -
     
    +
    +
    This method asynchronously sends a request to the replicas.
    +
    invokeOrdered(byte[]) - Method in class bftsmart.tom.ServiceProxy
    -
     
    +
    +
    This method sends an ordered request to the replicas, and returns the related reply.
    +
    invokeUnordered(byte[]) - Method in class bftsmart.tom.ServiceProxy
    -
     
    +
    +
    This method sends an unordered request to the replicas, and returns the related reply.
    +
    invokeUnorderedHashed(byte[]) - Method in class bftsmart.tom.ServiceProxy
    -
     
    +
    +
    This method sends an unordered request to the replicas, and returns the related reply.
    +
    isLastInBatch() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Returns the last request in the ordered batch
    +
    isNoOp() - Method in class bftsmart.tom.MessageContext
    -
     
    -
    isValidRequest(byte[]) - Method in interface bftsmart.tom.server.RequestVerifier
    -
     
    +
    +
    Returns true if the consensus instance has no operations to deliver to the application
    +
    +
    isValidRequest(TOMMessage) - Method in interface bftsmart.tom.server.RequestVerifier
    +
    +
    Given a request, validated it
    +
    @@ -385,16 +470,40 @@

    I

    J

    joinMsgReceived(VMMessage) - Method in class bftsmart.tom.ServiceReplica
    -
     
    +
    +
    Deprecated. 
    +
    +
    + + + +

    K

    +
    +
    KeyLoader - Interface in bftsmart.tom.util
    +
    +
    The KeyLoader interface is used internally by BFT-SMaRt to load signature keys from disk.
    +
    +
    kill() - Method in class bftsmart.tom.ServiceReplica
    +
    +
    Stops the service execution at a replica.
    +

    L

    -
    leave() - Method in class bftsmart.tom.ServiceReplica
    +
    loadPrivateKey() - Method in interface bftsmart.tom.util.KeyLoader
    +
    +
    Fetches the private key for this replica/client.
    +
    +
    loadPublicKey(int) - Method in interface bftsmart.tom.util.KeyLoader
    +
    +
    Fetches the public key for the specified replica/client.
    +
    +
    loadPublicKey() - Method in interface bftsmart.tom.util.KeyLoader
    -
    This method makes the replica leave the group
    +
    Fetches the public key for this replica/client.
    @@ -402,10 +511,10 @@

    L

    M

    -
    main(String[]) - Static method in class bftsmart.tom.util.RSAKeyPairGenerator
    -
     
    manageReply(TOMMessage, MessageContext) - Method in interface bftsmart.tom.server.Replier
    -
     
    +
    +
    Given an executed request, send it to a target
    +
    MessageContext - Class in bftsmart.tom
    This class represents the whole context of a request ordered in the system.
    @@ -420,13 +529,13 @@

    M

    N

    -
    noOp(int, MessageContext) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
    +
    noOp(int, byte[][], MessageContext[]) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
     
    -
    noOp(int, MessageContext) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    +
    noOp(int, byte[][], MessageContext[]) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
     
    -
    noOp(int, MessageContext) - Method in class bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator
    +
    noOp(int, byte[][], MessageContext[]) - Method in class bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator
     
    -
    noOp(int, MessageContext) - Method in interface bftsmart.tom.server.Recoverable
    +
    noOp(int, byte[][], MessageContext[]) - Method in interface bftsmart.tom.server.Recoverable
    This method is invoked by ServiceReplica to indicate that a consensus instance finished without delivering anything to the application (e.g., an instance @@ -458,11 +567,16 @@

    R

    readOnly - Variable in class bftsmart.tom.MessageContext
     
    receiveMessages(int[], int[], int[], CertifiedDecision[], TOMMessage[][]) - Method in class bftsmart.tom.ServiceReplica
    -
     
    +
    +
    Deprecated. 
    +
    receiveReadonlyMessage(TOMMessage, MessageContext) - Method in class bftsmart.tom.ServiceReplica
    -
    This message delivers a readonly message, i.e., a message that was not - ordered to the replica and gather the reply to forward to the client
    +
    Deprecated. 
    +
    +
    reconfigureTo(View) - Method in class bftsmart.tom.ServiceProxy
    +
    +
    Deprecated. 
    Recoverable - Interface in bftsmart.tom.server
    @@ -472,21 +586,29 @@

    R

    Generates a TOMMessage for its associated requests using the new info that it now supports since the previous commit.
    +
    removeServer(int) - Method in class bftsmart.reconfiguration.VMServices
    +
    +
    Removes a server from the group
    +
    ReplicaContext - Class in bftsmart.tom
    This class contains information related to the replica.
    ReplicaContext(ServerCommunicationSystem, ServerViewController) - Constructor for class bftsmart.tom.ReplicaContext
    -
     
    +
    +
    Constructor
    +
    replicaContext - Variable in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
     
    Replier - Interface in bftsmart.tom.server
    -
    Objects that implement this interface can be passed to the ServiceReplica - and manage replies in a custom manner.
    +
    Provides support for building custom reply management + to be used in the ServiceReplica.
    replyReceived(TOMMessage) - Method in class bftsmart.tom.AsynchServiceProxy
    -
     
    +
    +
    This is the method invoked by the client side communication system.
    +
    replyReceived(TOMMessage) - Method in class bftsmart.tom.ServiceProxy
    This is the method invoked by the client side communication system.
    @@ -495,27 +617,19 @@

    R

    This class contains information related to a client request.
    -
    RequestContext(int, int, TOMMessageType, int[], long, ReplyListener) - Constructor for class bftsmart.tom.RequestContext
    -
     
    +
    RequestContext(int, int, TOMMessageType, int[], long, ReplyListener, byte[]) - Constructor for class bftsmart.tom.RequestContext
    +
    +
    Constructor
    +
    RequestVerifier - Interface in bftsmart.tom.server
    Classes that implement this interface are invoked within consensus instances upon reception of a PROPOSE message in order to enforce the "external validity".
    -
    RSAKeyPairGenerator - Class in bftsmart.tom.util
    -
    -
    Utility class used to generate a key pair for some process id on - config/keys/publickey and config/keys/privatekey
    -
    -
    RSAKeyPairGenerator() - Constructor for class bftsmart.tom.util.RSAKeyPairGenerator
    +
    restart() - Method in class bftsmart.tom.ServiceReplica
    -
    Creates a new instance of KeyPairGenerator
    -
    -
    run(int, int) - Method in class bftsmart.tom.util.RSAKeyPairGenerator
    -
    -
    Generate the key pair for the process with id = and put it on the - files config/keys/publickey and config/keys/privatekey
    +
    Cleans the object state and reboots execution.
    @@ -523,14 +637,6 @@

    R

    S

    -
    saveCommands(byte[][], MessageContext[]) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
    -
    -
    Write commands to log file
    -
    -
    sendMessageToTargets(byte[], int, int[], TOMMessageType) - Method in class bftsmart.tom.TOMSender
    -
     
    -
    sendMessageToTargets(byte[], int, int, int[], TOMMessageType) - Method in class bftsmart.tom.TOMSender
    -
     
    ServiceProxy - Class in bftsmart.tom
    This class implements a TOMSender and represents a proxy to be used on the @@ -544,7 +650,11 @@

    S

    Constructor
    -
    ServiceProxy(int, String, Comparator<byte[]>, Extractor) - Constructor for class bftsmart.tom.ServiceProxy
    +
    ServiceProxy(int, String, KeyLoader) - Constructor for class bftsmart.tom.ServiceProxy
    +
    +
    Constructor
    +
    +
    ServiceProxy(int, String, Comparator<byte[]>, Extractor, KeyLoader) - Constructor for class bftsmart.tom.ServiceProxy
    Constructor
    @@ -561,25 +671,32 @@

    S

    Constructor
    -
    ServiceReplica(int, String, Executable, Recoverable, RequestVerifier) - Constructor for class bftsmart.tom.ServiceReplica
    +
    ServiceReplica(int, Executable, Recoverable, RequestVerifier, Replier) - Constructor for class bftsmart.tom.ServiceReplica
    Constructor
    -
    ServiceReplica(int, boolean, Executable, Recoverable) - Constructor for class bftsmart.tom.ServiceReplica
    +
    ServiceReplica(int, Executable, Recoverable, RequestVerifier, Replier, KeyLoader, Provider) - Constructor for class bftsmart.tom.ServiceReplica
    +
    +
    Constructor
    +
    +
    ServiceReplica(int, String, Executable, Recoverable, RequestVerifier, Replier, KeyLoader) - Constructor for class bftsmart.tom.ServiceReplica
    Constructor
    -
    ServiceReplica(int, String, boolean, Executable, Recoverable, RequestVerifier) - Constructor for class bftsmart.tom.ServiceReplica
    -
     
    setInvokeTimeout(int) - Method in class bftsmart.tom.ServiceProxy
    Set the amount of time (in seconds) that this proxy will wait for servers replies before returning null.
    setInvokeUnorderedHashedTimeout(int) - Method in class bftsmart.tom.ServiceProxy
    -
     
    +
    +
    Set the amount of time (in seconds) that this proxy will wait for + servers unordered hashed replies before returning null.
    +
    setLastInBatch() - Method in class bftsmart.tom.MessageContext
    -
     
    +
    +
    Deprecated. 
    +
    setReplicaContext(ReplicaContext) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
     
    setReplicaContext(ReplicaContext) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    @@ -587,13 +704,13 @@

    S

    setReplicaContext(ReplicaContext) - Method in class bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator
     
    setReplicaContext(ReplicaContext) - Method in interface bftsmart.tom.server.Recoverable
    -
     
    +
    +
    Sets the replica context
    +
    setReplicaContext(ReplicaContext) - Method in interface bftsmart.tom.server.Replier
    -
     
    -
    setReplyController(Replier) - Method in class bftsmart.tom.ServiceReplica
    -
     
    -
    setServerCommunicationSystem(ServerCommunicationSystem) - Method in class bftsmart.tom.ReplicaContext
    -
     
    +
    +
    Sets the replica context
    +
    setState(ApplicationState) - Method in class bftsmart.tom.server.defaultservices.DefaultRecoverable
     
    setState(ApplicationState) - Method in class bftsmart.tom.server.defaultservices.DefaultSingleRecoverable
    @@ -609,33 +726,25 @@

    S

    Executables that implement this interface will receive client requests individually.
    - + -

    T

    +

    V

    -
    TOMSender - Class in bftsmart.tom
    +
    VMServices - Class in bftsmart.reconfiguration
    -
    This class is used to multicast messages to replicas and receive replies.
    +
    This class is used by the trusted client to add and remove replicas from the group
    -
    TOMSender() - Constructor for class bftsmart.tom.TOMSender
    +
    VMServices() - Constructor for class bftsmart.reconfiguration.VMServices
    -
    Creates a new instance of TOMulticastSender - - TODO: This may really be empty?
    +
    Constructor.
    -
    TOMulticast(TOMMessage) - Method in class bftsmart.tom.TOMSender
    +
    VMServices(KeyLoader, String) - Constructor for class bftsmart.reconfiguration.VMServices
    -
    Multicast a TOMMessage to the group of replicas
    +
    Constructor.
    -
    TOMulticast(byte[], int, TOMMessageType) - Method in class bftsmart.tom.TOMSender
    -
    -
    Multicast data to the group of replicas
    -
    -
    TOMulticast(byte[], int, int, TOMMessageType) - Method in class bftsmart.tom.TOMSender
    -
     
    -A B C D E F G I J L M N O R S T 
    +A B C D E G I J K L M N O R S V 
    diff --git a/doc/index.html b/doc/index.html index a1e17b58e..cc8c2909b 100644 --- a/doc/index.html +++ b/doc/index.html @@ -2,14 +2,15 @@ - + Generated Documentation (Untitled) @@ -13,6 +13,7 @@

    Packages

      +
    • bftsmart.reconfiguration
    • bftsmart.tom
    • bftsmart.tom.server
    • bftsmart.tom.server.defaultservices
      • diff --git a/doc/overview-summary.html b/doc/overview-summary.html index b924b43e7..2897ca0fc 100644 --- a/doc/overview-summary.html +++ b/doc/overview-summary.html @@ -2,9 +2,9 @@ - + Overview - + @@ -77,22 +77,26 @@ -bftsmart.tom +bftsmart.reconfiguration   -bftsmart.tom.server +bftsmart.tom   -bftsmart.tom.server.defaultservices +bftsmart.tom.server   -bftsmart.tom.server.defaultservices.durability +bftsmart.tom.server.defaultservices   +bftsmart.tom.server.defaultservices.durability +  + + bftsmart.tom.util   diff --git a/doc/overview-tree.html b/doc/overview-tree.html index d5aeea495..b2bfc53b6 100644 --- a/doc/overview-tree.html +++ b/doc/overview-tree.html @@ -2,9 +2,9 @@ - + Class Hierarchy - + @@ -72,6 +72,7 @@

      Hierarchy For All Packages

      Package Hierarchies: @@ -109,15 +110,11 @@

      Interface Hierarchy

    • bftsmart.tom.server.Executable
    • bftsmart.tom.util.Extractor
      • +
    • bftsmart.tom.util.KeyLoader
    • bftsmart.tom.server.Recoverable
    • bftsmart.tom.server.Replier
    • bftsmart.tom.server.RequestVerifier
      • diff --git a/doc/package-list b/doc/package-list index bd82664b9..4526acd34 100644 --- a/doc/package-list +++ b/doc/package-list @@ -1,3 +1,4 @@ +bftsmart.reconfiguration bftsmart.tom bftsmart.tom.server bftsmart.tom.server.defaultservices diff --git a/doc/serialized-form.html b/doc/serialized-form.html index 3c0e058d3..3ee70fcbb 100644 --- a/doc/serialized-form.html +++ b/doc/serialized-form.html @@ -2,9 +2,9 @@ - + Serialized Form - + @@ -74,6 +74,139 @@

      Serialized Form

      • +

        Package bftsmart.reconfiguration

        +
          +
        • + + +

          Class bftsmart.reconfiguration.ReconfigureReply extends java.lang.Object implements Serializable

          +
            +
          • +

            Serialization Methods

            +
              +
            • +

              readExternal

              +
              public void readExternal(java.io.ObjectInput in)
+                  throws java.io.IOException,
+                         java.lang.ClassNotFoundException
              +
              +
              Throws:
              +
              java.io.IOException
              +
              java.lang.ClassNotFoundException
              +
              +
              • +
            • +

              writeExternal

              +
              public void writeExternal(java.io.ObjectOutput out)
+                   throws java.io.IOException
              +
              +
              Throws:
              +
              java.io.IOException
              +
              +
              • +
            +
            • +
          +
          • +
        • + + +

          Class bftsmart.reconfiguration.ReconfigureRequest extends java.lang.Object implements Serializable

          +
            +
          • +

            Serialization Methods

            +
              +
            • +

              readExternal

              +
              public void readExternal(java.io.ObjectInput in)
+                  throws java.io.IOException,
+                         java.lang.ClassNotFoundException
              +
              +
              Throws:
              +
              java.io.IOException
              +
              java.lang.ClassNotFoundException
              +
              +
              • +
            • +

              writeExternal

              +
              public void writeExternal(java.io.ObjectOutput out)
+                   throws java.io.IOException
              +
              +
              Throws:
              +
              java.io.IOException
              +
              +
              • +
            +
            • +
          +
          • +
        • + + +

          Class bftsmart.reconfiguration.View extends java.lang.Object implements Serializable

          +
          +
          serialVersionUID:
          +
          1466870385442069307L
          +
          +
            +
          • +

            Serialized Fields

            +
              +
            • +

              id

              +
              int id
              +
              • +
            • +

              f

              +
              int f
              +
              • +
            • +

              processes

              +
              int[] processes
              +
              • +
            • +

              addresses

              +
              java.util.Map<K,V> addresses
              +
              • +
            +
            • +
          +
          • +
        • + + +

          Class bftsmart.reconfiguration.VMMessage extends bftsmart.communication.SystemMessage implements Serializable

          +
            +
          • +

            Serialization Methods

            +
              +
            • +

              readExternal

              +
              public void readExternal(java.io.ObjectInput in)
+                  throws java.io.IOException,
+                         java.lang.ClassNotFoundException
              +
              +
              Throws:
              +
              java.io.IOException
              +
              java.lang.ClassNotFoundException
              +
              +
              • +
            • +

              writeExternal

              +
              public void writeExternal(java.io.ObjectOutput out)
+                   throws java.io.IOException
              +
              +
              Throws:
              +
              java.io.IOException
              +
              +
              • +
            +
            • +
          +
          • +
        +
        • +

      • Package bftsmart.tom

        • @@ -187,27 +320,15 @@

          Class bftsmart.tom.server.defaultservices.CommandsInfo extends java.lang.Obj
          • -

            Serialization Methods

            +

            Serialized Fields
            • diff --git a/lib/commons-codec-1.11.jar b/lib/commons-codec-1.11.jar new file mode 100644 index 000000000..22451206d Binary files /dev/null and b/lib/commons-codec-1.11.jar differ diff --git a/lib/commons-codec-1.5.jar b/lib/commons-codec-1.5.jar deleted file mode 100644 index e9013fed7..000000000 Binary files a/lib/commons-codec-1.5.jar and /dev/null differ diff --git a/lib/netty-all-4.1.34.Final.jar b/lib/netty-all-4.1.34.Final.jar new file mode 100644 index 000000000..ceecec79f Binary files /dev/null and b/lib/netty-all-4.1.34.Final.jar differ diff --git a/lib/netty-all-4.1.9.Final.jar b/lib/netty-all-4.1.9.Final.jar deleted file mode 100644 index 67df93166..000000000 Binary files a/lib/netty-all-4.1.9.Final.jar and /dev/null differ diff --git a/runscripts/generate-javadoc.sh b/runscripts/generate-javadoc.sh new file mode 100755 index 000000000..03d37ce8a --- /dev/null +++ b/runscripts/generate-javadoc.sh @@ -0,0 +1,2 @@ +rm -r doc/* +javadoc -cp src -d doc bftsmart.tom bftsmart.tom.server src/bftsmart/tom/util/KeyLoader.java src/bftsmart/tom/util/Extractor.java src/bftsmart/tom/server/defaultservices/DefaultRecoverable.java src/bftsmart/tom/server/defaultservices/DefaultSingleRecoverable.java src/bftsmart/tom/server/defaultservices/durability/DurabilityCoordinator.java src/bftsmart/reconfiguration/VMServices.java diff --git a/runscripts/smartrun.sh b/runscripts/smartrun.sh index 57af542b2..f302a10c7 100755 --- a/runscripts/smartrun.sh +++ b/runscripts/smartrun.sh @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -java -Dlogback.configurationFile="./config/logback.xml" -cp bin/*:lib/* $@ +java -Djava.security.properties="./config/java.security" -Dlogback.configurationFile="./config/logback.xml" -cp bin/*:lib/* $@ diff --git a/runscripts/startReplicaYCSB.sh b/runscripts/startReplicaYCSB.sh index fc23b9a7a..2df576987 100755 --- a/runscripts/startReplicaYCSB.sh +++ b/runscripts/startReplicaYCSB.sh @@ -16,4 +16,4 @@ REPLICA_INDEX=$1 -java -cp bin/:lib/* bftsmart.demo.ycsb.YCSBServer $REPLICA_INDEX +java -Dlogback.configurationFile="./config/logback.xml" -cp bin/:lib/* bftsmart.demo.ycsb.YCSBServer $REPLICA_INDEX diff --git a/runscripts/ycsbClient.sh b/runscripts/ycsbClient.sh index 5037b5be6..f22541fb5 100755 --- a/runscripts/ycsbClient.sh +++ b/runscripts/ycsbClient.sh @@ -13,4 +13,4 @@ # limitations under the License. #/bin/bash -java -cp ./lib/*:./bin/ com.yahoo.ycsb.Client -threads 10 -P config/workloads/workloada -p measurementtype=timeseries -p timeseries.granularity=1000 -db bftsmart.demo.ycsb.YCSBClient -s > output.txt +java -Dlogback.configurationFile="./config/logback.xml" -cp ./lib/*:./bin/ com.yahoo.ycsb.Client -threads 10 -P config/workloads/workloada -p measurementtype=timeseries -p timeseries.granularity=1000 -db bftsmart.demo.ycsb.YCSBClient -s diff --git a/src/bftsmart/clientsmanagement/ClientData.java b/src/bftsmart/clientsmanagement/ClientData.java index 73055ca32..ffd4cf302 100644 --- a/src/bftsmart/clientsmanagement/ClientData.java +++ b/src/bftsmart/clientsmanagement/ClientData.java @@ -41,7 +41,7 @@ public class ClientData { private int lastMessageReceived = -1; private long lastMessageReceivedTime = 0; - private int lastMessageExecuted = -1; + private int lastMessageDelivered = -1; private RequestList pendingRequests = new RequestList(); //anb: new code to deal with client requests that arrive after their execution @@ -89,12 +89,12 @@ public RequestList getOrderedRequests() { return orderedRequests; } - public void setLastMessageExecuted(int lastMessageExecuted) { - this.lastMessageExecuted = lastMessageExecuted; + public void setLastMessageDelivered(int lastMessageDelivered) { + this.lastMessageDelivered = lastMessageDelivered; } - public int getLastMessageExecuted() { - return lastMessageExecuted; + public int getLastMessageDelivered() { + return lastMessageDelivered; } public void setLastMessageReceived(int lastMessageReceived) { @@ -134,7 +134,7 @@ public boolean removeOrderedRequest(TOMMessage request) { } public boolean removeRequest(TOMMessage request) { - lastMessageExecuted = request.getSequence(); + lastMessageDelivered = request.getSequence(); boolean result = pendingRequests.remove(request); //anb: new code to deal with client requests that arrive after their execution orderedRequests.addLast(request); diff --git a/src/bftsmart/clientsmanagement/ClientsManager.java b/src/bftsmart/clientsmanagement/ClientsManager.java index 4a1111dde..a6db440db 100644 --- a/src/bftsmart/clientsmanagement/ClientsManager.java +++ b/src/bftsmart/clientsmanagement/ClientsManager.java @@ -25,6 +25,7 @@ import bftsmart.tom.core.messages.TOMMessage; import bftsmart.tom.leaderchange.RequestsTimer; import bftsmart.tom.server.RequestVerifier; +import bftsmart.tom.util.TOMUtil; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -67,7 +68,7 @@ public ClientData getClientData(int clientId) { //******* EDUARDO BEGIN **************// clientData = new ClientData(clientId, - (controller.getStaticConf().getUseSignatures() == 1) + (controller.getStaticConf().getUseSignatures()) ? controller.getStaticConf().getPublicKey(clientId) : null); //******* EDUARDO END **************// @@ -88,15 +89,18 @@ public ClientData getClientData(int clientId) { */ public RequestList getPendingRequests() { RequestList allReq = new RequestList(); - + clientsLock.lock(); /******* BEGIN CLIENTS CRITICAL SECTION ******/ Set> clientsEntrySet = clientsData.entrySet(); + logger.debug("Number of active clients: {}", clientsEntrySet.size()); for (int i = 0; true; i++) { Iterator> it = clientsEntrySet.iterator(); int noMoreMessages = 0; + + logger.debug("Fetching requests with internal index {}", i); while (it.hasNext() && allReq.size() < controller.getStaticConf().getMaxBatchSize() @@ -106,6 +110,9 @@ public RequestList getPendingRequests() { RequestList clientPendingRequests = clientData.getPendingRequests(); clientData.clientLock.lock(); + + logger.debug("Number of pending requests for client {}: {}.", clientData.getClientId(), clientPendingRequests.size()); + /******* BEGIN CLIENTDATA CRITICAL SECTION ******/ TOMMessage request = (clientPendingRequests.size() > i) ? clientPendingRequests.get(i) : null; @@ -114,6 +121,9 @@ public RequestList getPendingRequests() { if (request != null) { if(!request.alreadyProposed) { + + logger.debug("Selected request with sequence number {} from client {}", request.getSequence(), request.getSender()); + //this client have pending message request.alreadyProposed = true; allReq.addLast(request); @@ -170,6 +180,38 @@ public boolean havePendingRequests() { clientsLock.unlock(); return havePending; } + + /** + * Retrieves the number of pending requests + * @return Number of pending requests + */ + public int countPendingRequests() { + int count = 0; + + clientsLock.lock(); + /******* BEGIN CLIENTS CRITICAL SECTION ******/ + + Iterator> it = clientsData.entrySet().iterator(); + + while (it.hasNext()) { + ClientData clientData = it.next().getValue(); + + clientData.clientLock.lock(); + RequestList reqs = clientData.getPendingRequests(); + if (!reqs.isEmpty()) { + for(TOMMessage msg:reqs) { + if(!msg.alreadyProposed) { + count++; + } + } + } + clientData.clientLock.unlock(); + } + + /******* END CLIENTS CRITICAL SECTION ******/ + clientsLock.unlock(); + return count; + } /** * Verifies if some reqId is pending. @@ -219,19 +261,28 @@ public boolean requestReceived(TOMMessage request, boolean fromClient) { */ public boolean requestReceived(TOMMessage request, boolean fromClient, ServerCommunicationSystem cs) { - // if the content of the request is invalid, ignore it - if (controller.getStaticConf().isBFT() && !verifier.isValidRequest(request)) return false; + long receptionTime = System.nanoTime(); + long receptionTimestamp = System.currentTimeMillis(); - request.receptionTime = System.nanoTime(); - int clientId = request.getSender(); boolean accounted = false; - //Logger.println("(ClientsManager.requestReceived) getting info about client "+clientId); ClientData clientData = getClientData(clientId); - //Logger.println("(ClientsManager.requestReceived) wait for lock for client "+clientData.getClientId()); clientData.clientLock.lock(); + + //Is this a leader replay attack? + if (!fromClient && clientData.getSession() == request.getSession() && + clientData.getLastMessageDelivered() >= request.getSequence()) { + + clientData.clientLock.unlock(); + logger.warn("Detected a leader replay attack, rejecting request"); + return false; + } + + request.receptionTime = receptionTime; + request.receptionTimestamp = receptionTimestamp; + /******* BEGIN CLIENTDATA CRITICAL SECTION ******/ //Logger.println("(ClientsManager.requestReceived) lock for client "+clientData.getClientId()+" acquired"); @@ -255,6 +306,7 @@ public boolean requestReceived(TOMMessage request, boolean fromClient, ServerCom if (clientData.getSession() != request.getSession()) { clientData.setSession(request.getSession()); clientData.setLastMessageReceived(-1); + clientData.setLastMessageDelivered(-1); clientData.getOrderedRequests().clear(); clientData.getPendingRequests().clear(); } @@ -263,10 +315,18 @@ public boolean requestReceived(TOMMessage request, boolean fromClient, ServerCom (clientData.getLastMessageReceived() + 1 == request.getSequence()) || //message received is the expected ((request.getSequence() > clientData.getLastMessageReceived()) && !fromClient)) { - //it is a new message and I have to verify it's signature - if (!request.signed - || clientData.verifySignature(request.serializedMessage, - request.serializedMessageSignature)) { + //enforce the "external validity" property, i.e, verify if the + //requests are valid in accordance to the application semantics + //and not an erroneous requests sent by a Byzantine leader. + boolean isValid = (!controller.getStaticConf().isBFT() || verifier.isValidRequest(request)); + + //it is a valid new message and I have to verify it's signature + if (isValid && + (!request.signed || + clientData.verifySignature(request.serializedMessage, + request.serializedMessageSignature))) { + + logger.debug("Message from client {} is valid", clientData.getClientId()); //I don't have the message but it is valid, I will //insert it in the pending requests of this client @@ -282,17 +342,20 @@ public boolean requestReceived(TOMMessage request, boolean fromClient, ServerCom } accounted = true; + } else { + + logger.warn("Message from client {} is invalid", clientData.getClientId()); } } else { //I will not put this message on the pending requests list if (clientData.getLastMessageReceived() >= request.getSequence()) { //I already have/had this message - + //send reply if it is available TOMMessage reply = clientData.getReply(request.getSequence()); if (reply != null && cs != null) { - + if (reply.recvFromClient && fromClient) { logger.info("[CACHE] re-send reply [Sender: " + reply.getSender() + ", sequence: " + reply.getSequence()+", session: " + reply.getSession()+ "]"); cs.send(new int[]{request.getSender()}, reply); @@ -306,12 +369,16 @@ else if (!reply.recvFromClient && fromClient) { } accounted = true; } else { + + logger.warn("Message from client {} is too forward", clientData.getClientId()); + //a too forward message... the client must be malicious accounted = false; } } /******* END CLIENTDATA CRITICAL SECTION ******/ + clientData.clientLock.unlock(); return accounted; @@ -351,7 +418,7 @@ private void requestOrdered(TOMMessage request) { if (!clientData.removeOrderedRequest(request)) { logger.debug("Request " + request + " does not exist in pending requests"); } - clientData.setLastMessageExecuted(request.getSequence()); + clientData.setLastMessageDelivered(request.getSequence()); /******* END CLIENTDATA CRITICAL SECTION ******/ clientData.clientLock.unlock(); @@ -368,4 +435,9 @@ public void clear() { logger.info("ClientsManager cleared."); } + + public int numClients() { + + return clientsData.size(); + } } diff --git a/src/bftsmart/communication/MessageHandler.java b/src/bftsmart/communication/MessageHandler.java index e18ddd85d..4b6a97b32 100644 --- a/src/bftsmart/communication/MessageHandler.java +++ b/src/bftsmart/communication/MessageHandler.java @@ -15,27 +15,17 @@ */ package bftsmart.communication; -import bftsmart.communication.server.ServerConnection; -import bftsmart.consensus.messages.MessageFactory; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + import bftsmart.consensus.messages.ConsensusMessage; import bftsmart.consensus.roles.Acceptor; import bftsmart.statemanagement.SMMessage; import bftsmart.tom.core.TOMLayer; -import bftsmart.tom.core.messages.TOMMessage; import bftsmart.tom.core.messages.ForwardedMessage; +import bftsmart.tom.core.messages.TOMMessage; import bftsmart.tom.leaderchange.LCMessage; import bftsmart.tom.util.TOMUtil; -import java.io.ByteArrayOutputStream; -import java.io.IOException; -import java.io.ObjectOutputStream; -import java.security.InvalidKeyException; -import java.security.NoSuchAlgorithmException; -import java.util.Arrays; -import java.util.HashMap; -import javax.crypto.Mac; -import javax.crypto.SecretKey; -import org.slf4j.LoggerFactory; -import org.slf4j.Logger; /** * @@ -43,144 +33,105 @@ */ public class MessageHandler { - private Logger logger = LoggerFactory.getLogger(this.getClass()); - - private Acceptor acceptor; - private TOMLayer tomLayer; - //private Cipher cipher; - private Mac mac; - - public MessageHandler() { - try { - this.mac = TOMUtil.getMacFactory(); - } catch (NoSuchAlgorithmException /*| NoSuchPaddingException*/ ex) { - logger.error("Failed to create MAC engine",ex); - } - } - public void setAcceptor(Acceptor acceptor) { - this.acceptor = acceptor; - } - - public void setTOMLayer(TOMLayer tomLayer) { - this.tomLayer = tomLayer; - } - - @SuppressWarnings("unchecked") - protected void processData(SystemMessage sm) { - if (sm instanceof ConsensusMessage) { - - int myId = tomLayer.controller.getStaticConf().getProcessId(); - - ConsensusMessage consMsg = (ConsensusMessage) sm; - - if (tomLayer.controller.getStaticConf().getUseMACs() == 0 || consMsg.authenticated || consMsg.getSender() == myId) acceptor.deliver(consMsg); - else if (consMsg.getType() == MessageFactory.ACCEPT && consMsg.getProof() != null) { - - //We are going to verify the MAC vector at the algorithm level - HashMap macVector = (HashMap) consMsg.getProof(); - - byte[] recvMAC = macVector.get(myId); - - ConsensusMessage cm = new ConsensusMessage(MessageFactory.ACCEPT,consMsg.getNumber(), - consMsg.getEpoch(), consMsg.getSender(), consMsg.getValue()); - - ByteArrayOutputStream bOut = new ByteArrayOutputStream(248); - try { - new ObjectOutputStream(bOut).writeObject(cm); - } catch (IOException ex) { - logger.error("Failed to serialize consensus message",ex); - } - - byte[] data = bOut.toByteArray(); - - //byte[] hash = tomLayer.computeHash(data); - - byte[] myMAC = null; - - /*byte[] k = tomLayer.getCommunication().getServersConn().getSecretKey(paxosMsg.getSender()).getEncoded(); - SecretKeySpec key = new SecretKeySpec(new String(k).substring(0, 8).getBytes(), "DES");*/ - - SecretKey key = tomLayer.getCommunication().getServersConn().getSecretKey(consMsg.getSender()); - try { - this.mac.init(key); - myMAC = this.mac.doFinal(data); - } catch (/*IllegalBlockSizeException | BadPaddingException |*/ InvalidKeyException ex) { - logger.error("Failed to generate MAC",ex); - } - - if (recvMAC != null && myMAC != null && Arrays.equals(recvMAC, myMAC)) - acceptor.deliver(consMsg); - else { - logger.warn("Invalid MAC from " + sm.getSender()); - } - } else { - logger.warn("Discarding unauthenticated message from " + sm.getSender()); - } - - } else { - if (tomLayer.controller.getStaticConf().getUseMACs() == 0 || sm.authenticated) { - /*** This is Joao's code, related to leader change */ - if (sm instanceof LCMessage) { - LCMessage lcMsg = (LCMessage) sm; - - String type = null; - switch(lcMsg.getType()) { - - case TOMUtil.STOP: - type = "STOP"; - break; - case TOMUtil.STOPDATA: - type = "STOPDATA"; - break; - case TOMUtil.SYNC: - type = "SYNC"; - break; - default: - type = "LOCAL"; - break; - } - - logger.info("LC_MSG received: type " + type + ", regency " + lcMsg.getReg() + ", (replica " + lcMsg.getSender() + ")"); - if (lcMsg.TRIGGER_LC_LOCALLY) tomLayer.requestsTimer.run_lc_protocol(); - else tomLayer.getSynchronizer().deliverTimeoutRequest(lcMsg); - /**************************************************************/ - - } else if (sm instanceof ForwardedMessage) { - TOMMessage request = ((ForwardedMessage) sm).getRequest(); - tomLayer.requestReceived(request); - - /** This is Joao's code, to handle state transfer */ - } else if (sm instanceof SMMessage) { - SMMessage smsg = (SMMessage) sm; - // System.out.println("(MessageHandler.processData) SM_MSG received: type " + smsg.getType() + ", regency " + smsg.getRegency() + ", (replica " + smsg.getSender() + ")"); - switch(smsg.getType()) { - case TOMUtil.SM_REQUEST: - tomLayer.getStateManager().SMRequestDeliver(smsg, tomLayer.controller.getStaticConf().isBFT()); - break; - case TOMUtil.SM_REPLY: - tomLayer.getStateManager().SMReplyDeliver(smsg, tomLayer.controller.getStaticConf().isBFT()); - break; - case TOMUtil.SM_ASK_INITIAL: - tomLayer.getStateManager().currentConsensusIdAsked(smsg.getSender()); - break; - case TOMUtil.SM_REPLY_INITIAL: - tomLayer.getStateManager().currentConsensusIdReceived(smsg); - break; - default: - tomLayer.getStateManager().stateTimeout(); - break; - } - /******************************************************************/ - } else { - logger.warn("UNKNOWN MESSAGE TYPE: " + sm); - } - } else { - logger.warn("Discarding unauthenticated message from " + sm.getSender()); - } - } - } - - protected void verifyPending() { - tomLayer.processOutOfContext(); - } + private Logger logger = LoggerFactory.getLogger(this.getClass()); + + private Acceptor acceptor; + private TOMLayer tomLayer; + + public MessageHandler() {} + + public void setAcceptor(Acceptor acceptor) { + this.acceptor = acceptor; + } + + public void setTOMLayer(TOMLayer tomLayer) { + this.tomLayer = tomLayer; + } + + @SuppressWarnings("unchecked") + protected void processData(SystemMessage sm) { + if (sm instanceof ConsensusMessage) { + + int myId = tomLayer.controller.getStaticConf().getProcessId(); + + ConsensusMessage consMsg = (ConsensusMessage) sm; + + if (consMsg.authenticated || consMsg.getSender() == myId) + acceptor.deliver(consMsg); + else { + logger.warn("Discarding unauthenticated message from " + sm.getSender()); + } + + } else { + if (sm.authenticated) { + /*** This is Joao's code, related to leader change */ + if (sm instanceof LCMessage) { + LCMessage lcMsg = (LCMessage) sm; + + String type = null; + switch (lcMsg.getType()) { + + case TOMUtil.STOP: + type = "STOP"; + break; + case TOMUtil.STOPDATA: + type = "STOPDATA"; + break; + case TOMUtil.SYNC: + type = "SYNC"; + break; + default: + type = "LOCAL"; + break; + } + + if (lcMsg.getReg() != -1 && lcMsg.getSender() != -1) + logger.info("Received leader change message of type {} " + "for regency {} from replica {}", + type, lcMsg.getReg(), lcMsg.getSender()); + else + logger.debug("Received leader change message from myself"); + + if (lcMsg.TRIGGER_LC_LOCALLY) + tomLayer.requestsTimer.run_lc_protocol(); + else + tomLayer.getSynchronizer().deliverTimeoutRequest(lcMsg); + /**************************************************************/ + + } else if (sm instanceof ForwardedMessage) { + TOMMessage request = ((ForwardedMessage) sm).getRequest(); + tomLayer.requestReceived(request); + + /** This is Joao's code, to handle state transfer */ + } else if (sm instanceof SMMessage) { + SMMessage smsg = (SMMessage) sm; + switch (smsg.getType()) { + case TOMUtil.SM_REQUEST: + tomLayer.getStateManager().SMRequestDeliver(smsg, tomLayer.controller.getStaticConf().isBFT()); + break; + case TOMUtil.SM_REPLY: + tomLayer.getStateManager().SMReplyDeliver(smsg, tomLayer.controller.getStaticConf().isBFT()); + break; + case TOMUtil.SM_ASK_INITIAL: + tomLayer.getStateManager().currentConsensusIdAsked(smsg.getSender(), smsg.getCID()); + break; + case TOMUtil.SM_REPLY_INITIAL: + tomLayer.getStateManager().currentConsensusIdReceived(smsg); + break; + default: + tomLayer.getStateManager().stateTimeout(); + break; + } + /******************************************************************/ + } else { + logger.warn("UNKNOWN MESSAGE TYPE: " + sm); + } + } else { + logger.warn("Discarding unauthenticated message from " + sm.getSender()); + } + } + } + + protected void verifyPending() { + tomLayer.processOutOfContext(); + } } diff --git a/src/bftsmart/communication/ServerCommunicationSystem.java b/src/bftsmart/communication/ServerCommunicationSystem.java index abfd215fb..1464faaf8 100644 --- a/src/bftsmart/communication/ServerCommunicationSystem.java +++ b/src/bftsmart/communication/ServerCommunicationSystem.java @@ -18,6 +18,8 @@ import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.TimeUnit; +import javax.crypto.SecretKey; + import bftsmart.communication.client.CommunicationSystemServerSide; import bftsmart.communication.client.CommunicationSystemServerSideFactory; import bftsmart.communication.client.RequestReceiver; @@ -43,6 +45,7 @@ public class ServerCommunicationSystem extends Thread { public final long MESSAGE_WAIT_TIME = 100; private LinkedBlockingQueue inQueue = null;//new LinkedBlockingQueue(IN_QUEUE_SIZE); protected MessageHandler messageHandler; + private ServersCommunicationLayer serversConn; private CommunicationSystemServerSide clientsConn; private ServerViewController controller; @@ -51,7 +54,7 @@ public class ServerCommunicationSystem extends Thread { * Creates a new instance of ServerCommunicationSystem */ public ServerCommunicationSystem(ServerViewController controller, ServiceReplica replica) throws Exception { - super("Server CS"); + super("Server Comm. System"); this.controller = controller; @@ -59,20 +62,11 @@ public ServerCommunicationSystem(ServerViewController controller, ServiceReplica inQueue = new LinkedBlockingQueue(controller.getStaticConf().getInQueueSize()); - //create a new conf, with updated port number for servers - //TOMConfiguration serversConf = new TOMConfiguration(conf.getProcessId(), - // Configuration.getHomeDir(), "hosts.config"); - - //serversConf.increasePortNumber(); - serversConn = new ServersCommunicationLayer(controller, inQueue, replica); //******* EDUARDO BEGIN **************// - // if (manager.isInCurrentView() || manager.isInInitView()) { clientsConn = CommunicationSystemServerSideFactory.getCommunicationSystemServerSide(controller); - // } //******* EDUARDO END **************// - //start(); } //******* EDUARDO BEGIN **************// @@ -120,7 +114,7 @@ public void run() { SystemMessage sm = inQueue.poll(MESSAGE_WAIT_TIME, TimeUnit.MILLISECONDS); if (sm != null) { - logger.debug("<-------receiving---------- " + sm); + logger.debug("<-- receiving, msg:" + sm); messageHandler.processData(sm); count++; } else { @@ -147,7 +141,7 @@ public void send(int[] targets, SystemMessage sm) { if (sm instanceof TOMMessage) { clientsConn.send(targets, (TOMMessage) sm, false); } else { - logger.debug("--------sending----------> " + sm); + logger.debug("--> sending message from: {} -> {}" + sm.getSender(), targets); serversConn.send(targets, sm, true); } } @@ -173,4 +167,8 @@ public void shutdown() { clientsConn.shutdown(); serversConn.shutdown(); } + + public SecretKey getSecretKey(int id) { + return serversConn.getSecretKey(id); + } } diff --git a/src/bftsmart/communication/client/netty/NettyClientPipelineFactory.java b/src/bftsmart/communication/client/netty/NettyClientPipelineFactory.java index 9d444b179..9f54a6022 100644 --- a/src/bftsmart/communication/client/netty/NettyClientPipelineFactory.java +++ b/src/bftsmart/communication/client/netty/NettyClientPipelineFactory.java @@ -20,41 +20,38 @@ import io.netty.handler.codec.MessageToByteEncoder; import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.locks.ReentrantReadWriteLock; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import bftsmart.reconfiguration.ClientViewController; public class NettyClientPipelineFactory{ - + NettyClientServerCommunicationSystemClientSide ncs; - Map sessionTable; - int macLength; - int signatureLength; - - //******* EDUARDO BEGIN **************// + ConcurrentHashMap sessionTable; ClientViewController controller; - //******* EDUARDO END **************// - ReentrantReadWriteLock rl; - public NettyClientPipelineFactory(NettyClientServerCommunicationSystemClientSide ncs, Map sessionTable, int macLength, ClientViewController controller, ReentrantReadWriteLock rl, int signatureLength) { + public NettyClientPipelineFactory(NettyClientServerCommunicationSystemClientSide ncs, + ConcurrentHashMap sessionTable, ClientViewController controller, ReentrantReadWriteLock rl) { this.ncs = ncs; this.sessionTable = sessionTable; - this.macLength = macLength; - this.signatureLength = signatureLength; this.rl = rl; this.controller = controller; } public ByteToMessageDecoder getDecoder(){ - return new NettyTOMMessageDecoder(true, sessionTable, macLength,controller,rl,signatureLength,controller.getStaticConf().getUseMACs()==1?true:false); + return new NettyTOMMessageDecoder(true, sessionTable, + controller,rl); } public MessageToByteEncoder getEncoder(){ - return new NettyTOMMessageEncoder(true, sessionTable, macLength,rl, signatureLength, controller.getStaticConf().getUseMACs()==1?true:false); + return new NettyTOMMessageEncoder(true, sessionTable,rl); } public SimpleChannelInboundHandler getHandler(){ diff --git a/src/bftsmart/communication/client/netty/NettyClientServerCommunicationSystemClientSide.java b/src/bftsmart/communication/client/netty/NettyClientServerCommunicationSystemClientSide.java index 3ba225e86..268ba4a8d 100644 --- a/src/bftsmart/communication/client/netty/NettyClientServerCommunicationSystemClientSide.java +++ b/src/bftsmart/communication/client/netty/NettyClientServerCommunicationSystemClientSide.java @@ -15,21 +15,6 @@ */ package bftsmart.communication.client.netty; -import io.netty.bootstrap.Bootstrap; -import io.netty.channel.Channel; -import io.netty.channel.ChannelFuture; -import io.netty.channel.ChannelHandler.Sharable; -import io.netty.channel.ChannelHandlerContext; -import io.netty.channel.ChannelInitializer; -import io.netty.channel.ChannelOption; -import io.netty.channel.EventLoop; -import io.netty.channel.EventLoopGroup; -import io.netty.channel.SimpleChannelInboundHandler; -import io.netty.channel.nio.NioEventLoopGroup; -import io.netty.channel.socket.SocketChannel; -import io.netty.channel.socket.nio.NioSocketChannel; -import io.netty.util.concurrent.GenericFutureListener; - import java.io.ByteArrayOutputStream; import java.io.DataOutputStream; import java.io.IOException; @@ -41,531 +26,512 @@ import java.security.Signature; import java.security.spec.InvalidKeySpecException; import java.util.ArrayList; -import java.util.HashMap; -import java.util.Map; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.locks.ReentrantReadWriteLock; import java.util.Arrays; +import java.util.Collections; +import java.util.Random; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.TimeUnit; import java.util.concurrent.locks.Condition; import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; +import java.util.concurrent.locks.ReentrantReadWriteLock; -import javax.crypto.Mac; import javax.crypto.SecretKey; import javax.crypto.SecretKeyFactory; import javax.crypto.spec.PBEKeySpec; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + import bftsmart.communication.client.CommunicationSystemClientSide; import bftsmart.communication.client.ReplyReceiver; import bftsmart.reconfiguration.ClientViewController; import bftsmart.tom.core.messages.TOMMessage; import bftsmart.tom.util.TOMUtil; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; +import io.netty.bootstrap.Bootstrap; +import io.netty.channel.Channel; +import io.netty.channel.ChannelFuture; +import io.netty.channel.ChannelHandler.Sharable; +import io.netty.channel.ChannelHandlerContext; +import io.netty.channel.ChannelInitializer; +import io.netty.channel.ChannelOption; +import io.netty.channel.EventLoop; +import io.netty.channel.EventLoopGroup; +import io.netty.channel.SimpleChannelInboundHandler; +import io.netty.channel.nio.NioEventLoopGroup; +import io.netty.channel.socket.SocketChannel; +import io.netty.channel.socket.nio.NioSocketChannel; +import io.netty.util.concurrent.GenericFutureListener; /** * * @author Paulo */ @Sharable -public class NettyClientServerCommunicationSystemClientSide extends SimpleChannelInboundHandler implements CommunicationSystemClientSide { - - private Logger logger = LoggerFactory.getLogger(this.getClass()); - - private int clientId; - protected ReplyReceiver trr; - //******* EDUARDO BEGIN **************// - private ClientViewController controller; - //******* EDUARDO END **************// - private Map sessionTable = new HashMap<>(); - private ReentrantReadWriteLock rl; - //the signature engine used in the system - private Signature signatureEngine; - private int signatureLength; - private boolean closed = false; - - private EventLoopGroup workerGroup; - - private SyncListener listener; - - public NettyClientServerCommunicationSystemClientSide(int clientId, ClientViewController controller) { - super(); - - this.clientId = clientId; - this.workerGroup = new NioEventLoopGroup(); - try { - SecretKeyFactory fac = TOMUtil.getSecretFactory(); - - this.controller = controller; - this.listener = new SyncListener(); - - //this.st = new Storage(BENCHMARK_PERIOD); - this.rl = new ReentrantReadWriteLock(); - signatureLength = TOMUtil.getSignatureSize(controller); - - ChannelFuture future = null; - int[] currV = controller.getCurrentViewProcesses(); - for (int i = 0; i < currV.length; i++) { - try { - - String str = this.clientId + ":" + currV[i]; - PBEKeySpec spec = new PBEKeySpec(str.toCharArray()); - SecretKey authKey = fac.generateSecret(spec); - - //EventLoopGroup workerGroup = new NioEventLoopGroup(); - - //try { - Bootstrap b = new Bootstrap(); - b.group(workerGroup); - b.channel(NioSocketChannel.class); - b.option(ChannelOption.SO_KEEPALIVE, true); - b.option(ChannelOption.TCP_NODELAY, true); - b.option(ChannelOption.CONNECT_TIMEOUT_MILLIS,10000); - - b.handler(getChannelInitializer()); - - // Start the client. - future = b.connect(controller.getRemoteAddress(currV[i])); - - //******* EDUARDO BEGIN **************// - - //creates MAC stuff - Mac macSend = TOMUtil.getMacFactory(); - macSend.init(authKey); - Mac macReceive = TOMUtil.getMacFactory(); - macReceive.init(authKey); - NettyClientServerSession cs = new NettyClientServerSession(future.channel(), macSend, macReceive, currV[i]); - sessionTable.put(currV[i], cs); - - logger.info("Connecting to replica " + currV[i] + " at " + controller.getRemoteAddress(currV[i])); - //******* EDUARDO END **************// - - future.awaitUninterruptibly(); - - if (!future.isSuccess()) { - logger.error("Impossible to connect to " + currV[i]); - } - - } catch (java.lang.NullPointerException ex) { - //What is this??? This is not possible!!! - logger.debug("Should fix the problem, and I think it has no other implications :-), " - + "but we must make the servers store the view in a different place."); - } catch (InvalidKeyException ex) { - logger.error("Failed to initialize MAC engine",ex); - } catch (Exception ex){ - logger.error("Failed to initialize MAC engine",ex); - } - } - } catch (NoSuchAlgorithmException ex) { - logger.error("Failed to initialize secret key factory",ex); - } - } - - @Override - public void updateConnections() { - int[] currV = controller.getCurrentViewProcesses(); - try { - //open connections with new servers - for (int i = 0; i < currV.length; i++) { - rl.readLock().lock(); - if (sessionTable.get(currV[i]) == null) { - - rl.readLock().unlock(); - rl.writeLock().lock(); - - SecretKeyFactory fac = TOMUtil.getSecretFactory(); - try { - // Configure the client. - - //EventLoopGroup workerGroup = new NioEventLoopGroup(); - if( workerGroup == null){ - workerGroup = new NioEventLoopGroup(); - } - - //try { - Bootstrap b = new Bootstrap(); - b.group(workerGroup); - b.channel(NioSocketChannel.class); - b.option(ChannelOption.SO_KEEPALIVE, true); - b.option(ChannelOption.TCP_NODELAY, true); - b.option(ChannelOption.CONNECT_TIMEOUT_MILLIS,10000); - - b.handler(getChannelInitializer()); - - // Start the client. - ChannelFuture future = b.connect(controller.getRemoteAddress(currV[i])); - - String str = this.clientId + ":" + currV[i]; - PBEKeySpec spec = new PBEKeySpec(str.toCharArray()); - SecretKey authKey = fac.generateSecret(spec); - - //creates MAC stuff - Mac macSend = TOMUtil.getMacFactory(); - macSend.init(authKey); - Mac macReceive = TOMUtil.getMacFactory(); - macReceive.init(authKey); - NettyClientServerSession cs = new NettyClientServerSession(future.channel(), macSend, macReceive, currV[i]); - sessionTable.put(currV[i], cs); - - logger.info("Connecting to replica " + currV[i] + " at " + controller.getRemoteAddress(currV[i])); - //******* EDUARDO END **************// - - future.awaitUninterruptibly(); - - if (!future.isSuccess()) { - logger.error("Impossible to connect to " + currV[i]); - } - - } catch (InvalidKeyException | InvalidKeySpecException ex) { - logger.error("Failed to initialize MAC engine",ex); - } - rl.writeLock().unlock(); - } else { - rl.readLock().unlock(); - } - } - } catch (NoSuchAlgorithmException ex) { - logger.error("Failed to initialzie secret key factory",ex); - } - } - - @Override - public void exceptionCaught(ChannelHandlerContext ctx,Throwable cause) throws Exception { - if(cause instanceof ClosedChannelException) { - logger.error("Connection with replica closed.",cause); - } else if(cause instanceof ConnectException) { - logger.error("Impossible to connect to replica.",cause); - } else { - logger.error("Replica disconnected.",cause); - } - } - - @Override - public void channelRead0(ChannelHandlerContext ctx, TOMMessage sm) throws Exception { - - if(closed){ - - closeChannelAndEventLoop(ctx.channel()); - - return; - } - trr.replyReceived(sm); - } - - @Override - public void channelActive(ChannelHandlerContext ctx) { - - if(closed){ - - closeChannelAndEventLoop(ctx.channel()); - - return; - } - - logger.info("Channel active"); - } - - public void reconnect(final ChannelHandlerContext ctx){ - - rl.writeLock().lock(); - logger.debug("try to reconnect"); - - //Iterator sessions = sessionTable.values().iterator(); - - ArrayList sessions = new ArrayList(sessionTable.values()); - for (NettyClientServerSession ncss : sessions) { - if (ncss.getChannel() == ctx.channel()) { - try { - // Configure the client. - //EventLoopGroup workerGroup = ctx.channel().eventLoop(); - if( workerGroup == null){ - workerGroup = new NioEventLoopGroup(); - } - - //try { - Bootstrap b = new Bootstrap(); - b.group(workerGroup); - b.channel(NioSocketChannel.class); - b.option(ChannelOption.SO_KEEPALIVE, true); - b.option(ChannelOption.TCP_NODELAY, true); - b.option(ChannelOption.CONNECT_TIMEOUT_MILLIS,10000); - - b.handler(getChannelInitializer()); - - if (controller.getRemoteAddress(ncss.getReplicaId()) != null) { - - ChannelFuture future = b.connect(controller.getRemoteAddress(ncss.getReplicaId())); - - //creates MAC stuff - Mac macSend = ncss.getMacSend(); - Mac macReceive = ncss.getMacReceive(); - NettyClientServerSession cs = new NettyClientServerSession(future.channel(), macSend, macReceive, ncss.getReplicaId()); - sessionTable.remove(ncss.getReplicaId()); - sessionTable.put(ncss.getReplicaId(), cs); - - logger.info("re-connecting to replica "+ncss.getReplicaId()+" at " + controller.getRemoteAddress(ncss.getReplicaId())); - } else { - // This cleans an olde server from the session table - sessionTable.remove(ncss.getReplicaId()); - } - } catch (NoSuchAlgorithmException ex) { - logger.error("Failed to reconnect to replica",ex); - } - } - } - - //closes all other channels to avoid messages being sent to only a subset of the replicas - /*Enumeration sessionElements = sessionTable.elements(); - while (sessionElements.hasMoreElements()){ - ((NettyClientServerSession) sessionElements.nextElement()).getChannel().close(); - }*/ - rl.writeLock().unlock(); - } - - @Override - public void setReplyReceiver(ReplyReceiver trr) { - this.trr = trr; - } - - @Override - public void send(boolean sign, int[] targets, TOMMessage sm) { - - int quorum; - - if (controller.getStaticConf().isBFT()) { - quorum = (int) Math.ceil((controller.getCurrentViewN() - + controller.getCurrentViewF()) / 2) + 1; - } else { - quorum = (int) Math.ceil((controller.getCurrentViewN()) / 2) + 1; - } - - listener.waitForChannels(quorum); // wait for the previous transmission to complete - - logger.debug("Sending request from " + sm.getSender() + " with sequence number " + sm.getSequence() + " to " + Arrays.toString(targets)); - - if (sm.serializedMessage == null) { - - //serialize message - DataOutputStream dos = null; - try { - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - dos = new DataOutputStream(baos); - sm.wExternal(dos); - dos.flush(); - sm.serializedMessage = baos.toByteArray(); - } catch (IOException ex) { - logger.debug("Impossible to serialize message: " + sm); - } - } - - //Logger.println("Sending message with "+sm.serializedMessage.length+" bytes of content."); - - //produce signature - if (sign && sm.serializedMessageSignature == null) { - sm.serializedMessageSignature = signMessage( - controller.getStaticConf().getPrivateKey(), sm.serializedMessage); - } - - int sent = 0; - for (int i = targets.length - 1; i >= 0; i--) { - sm.destination = targets[i]; - - rl.readLock().lock(); - Channel channel = ((NettyClientServerSession) sessionTable.get(targets[i])).getChannel(); - rl.readLock().unlock(); - if (channel.isActive()) { - sm.signed = sign; - ChannelFuture f = channel.writeAndFlush(sm); - - f.addListener(listener); - - sent++; - } else { - logger.debug("Channel to " + targets[i] + " is not connected"); - } - - try { - sm = (TOMMessage) sm.clone(); - } catch (CloneNotSupportedException e) { - logger.error("Failed to clone TOMMessage",e); - } - } - - if (targets.length > controller.getCurrentViewF() && sent < controller.getCurrentViewF() + 1) { - //if less than f+1 servers are connected send an exception to the client - throw new RuntimeException("Impossible to connect to servers!"); - } - if(targets.length == 1 && sent == 0) - throw new RuntimeException("Server not connected"); - } - - public void sign(TOMMessage sm) { - //serialize message - DataOutputStream dos = null; - byte[] data = null; - try { - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - dos = new DataOutputStream(baos); - sm.wExternal(dos); - dos.flush(); - data = baos.toByteArray(); - sm.serializedMessage = data; - } catch (IOException ex) { - - logger.error("Failed to sign TOMMessage", ex); - } - - //******* EDUARDO BEGIN **************// - //produce signature - byte[] data2 = signMessage(controller.getStaticConf().getPrivateKey(), data); - //******* EDUARDO END **************// - - sm.serializedMessageSignature = data2; - } - - public byte[] signMessage(PrivateKey key, byte[] message) { - //long startTime = System.nanoTime(); - try { - if (signatureEngine == null) { - signatureEngine = TOMUtil.getSigEngine(); - } - byte[] result = null; - - signatureEngine.initSign(key); - signatureEngine.update(message); - result = signatureEngine.sign(); - - //st.store(System.nanoTime() - startTime); - return result; - } catch (Exception e) { - logger.error("Failed to sign message",e); - return null; - } - } - - @Override - public void close() { - this.closed = true; - //Iterator sessions = sessionTable.values().iterator(); - rl.readLock().lock(); - ArrayList sessions = new ArrayList<>(sessionTable.values()); - rl.readLock().unlock(); - for (NettyClientServerSession ncss : sessions) { - Channel c = ncss.getChannel(); - closeChannelAndEventLoop(c); - } - } - - private ChannelInitializer getChannelInitializer() throws NoSuchAlgorithmException{ - - Mac macDummy = TOMUtil.getMacFactory(); - - final NettyClientPipelineFactory nettyClientPipelineFactory = new NettyClientPipelineFactory(this, sessionTable, - macDummy.getMacLength(), controller, rl, signatureLength); - - ChannelInitializer channelInitializer = new ChannelInitializer() { - @Override - public void initChannel(SocketChannel ch) throws Exception { - ch.pipeline().addLast(nettyClientPipelineFactory.getDecoder()); - ch.pipeline().addLast(nettyClientPipelineFactory.getEncoder()); - ch.pipeline().addLast(nettyClientPipelineFactory.getHandler()); - - } - }; - return channelInitializer; - } - - @Override - public void channelUnregistered(final ChannelHandlerContext ctx) throws Exception { - scheduleReconnect(ctx,10); - } - - @Override - public void channelInactive(final ChannelHandlerContext ctx){ - scheduleReconnect(ctx,10); - } - - private void closeChannelAndEventLoop(Channel c) { - // once having an event in your handler (EchoServerHandler) - // Close the current channel - c.close(); - // Then close the parent channel (the one attached to the bind) - if (c.parent() != null) c.parent().close(); - //c.eventLoop().shutdownGracefully(); - workerGroup.shutdownGracefully(); - } - - private void scheduleReconnect(final ChannelHandlerContext ctx, int time){ - if(closed){ - closeChannelAndEventLoop(ctx.channel()); - return; - } - - final EventLoop loop = ctx.channel().eventLoop(); - loop.schedule(new Runnable() { - @Override - public void run() { - reconnect(ctx); - } - },time,TimeUnit.SECONDS); - } - - - private class SyncListener implements GenericFutureListener { - - private int remainingFutures; - - private final Lock futureLock; - private final Condition enoughCompleted; - - public SyncListener() { - - this.remainingFutures = 0; - - this.futureLock = new ReentrantLock(); - this.enoughCompleted = futureLock.newCondition(); - } - - @Override - public void operationComplete(ChannelFuture f) { - - this.futureLock.lock(); - - this.remainingFutures--; - - if (this.remainingFutures <= 0) { - - this.enoughCompleted.signalAll(); - } - - logger.debug(this.remainingFutures + " channel operations remaining to complete"); - - this.futureLock.unlock(); - - } - - public void waitForChannels(int n) { - - this.futureLock.lock(); - if (this.remainingFutures > 0) { - - logger.debug("There are still " + this.remainingFutures + " channel operations pending, waiting to complete"); - - try { - this.enoughCompleted.await(1000, TimeUnit.MILLISECONDS); // timeout if a malicous replica refuses to acknowledge the operation as completed - } catch (InterruptedException ex) { - logger.error("Interruption while waiting on condition", ex); - } - - } - - logger.debug("All channel operations completed or timed out"); - - this.remainingFutures = n; - - this.futureLock.unlock(); - } - - } +public class NettyClientServerCommunicationSystemClientSide extends SimpleChannelInboundHandler + implements CommunicationSystemClientSide { + + private Logger logger = LoggerFactory.getLogger(this.getClass()); + + private int clientId; + protected ReplyReceiver trr; + // ******* EDUARDO BEGIN **************// + private ClientViewController controller; + // ******* EDUARDO END **************// + private ConcurrentHashMap sessionClientToReplica = new ConcurrentHashMap<>(); + private ReentrantReadWriteLock rl; + private Signature signatureEngine; + private boolean closed = false; + + private EventLoopGroup workerGroup; + private SyncListener listener; + + private SecretKeyFactory secretKeyFactory; + + /* Tulio Ribeiro */ + private static int tcpSendBufferSize = 8 * 1024 * 1024; + private static int connectionTimeoutMsec = 40000; /* (40 seconds, timeout) */ + private PrivateKey privKey; + /* end Tulio Ribeiro */ + + public NettyClientServerCommunicationSystemClientSide(int clientId, ClientViewController controller) { + super(); + + this.clientId = clientId; + this.workerGroup = new NioEventLoopGroup(Runtime.getRuntime().availableProcessors()); + try { + + this.secretKeyFactory = TOMUtil.getSecretFactory(); + + this.controller = controller; + + /* Tulio Ribeiro */ + privKey = controller.getStaticConf().getPrivateKey(); + + this.listener = new SyncListener(); + this.rl = new ReentrantReadWriteLock(); + + int[] currV = controller.getCurrentViewProcesses(); + + for (int i = 0; i < currV.length; i++) { + int replicaId = currV[i]; + try { + + ChannelFuture future = connectToReplica(replicaId, secretKeyFactory); + + logger.debug("ClientID {}, connecting to replica {}, at address: {}", clientId, replicaId, + controller.getRemoteAddress(replicaId)); + + future.awaitUninterruptibly(); + + if (!future.isSuccess()) { + logger.error("Impossible to connect to " + replicaId); + } + + } catch (java.lang.NullPointerException ex) { + // What is this??? This is not possible!!! + logger.debug("Should fix the problem, and I think it has no other implications :-), " + + "but we must make the servers store the view in a different place."); + } catch (Exception ex) { + logger.error("Failed to initialize MAC engine", ex); + } + } + } catch (NoSuchAlgorithmException ex) { + logger.error("Failed to initialize secret key factory", ex); + } + } + + @Override + public void updateConnections() { + int[] currV = controller.getCurrentViewProcesses(); + try { + // open connections with new servers + for (int i = 0; i < currV.length; i++) { + + int replicaId = currV[i]; + + rl.readLock().lock(); + if (sessionClientToReplica.get(replicaId) == null) { + rl.readLock().unlock(); + rl.writeLock().lock(); + try { + ChannelFuture future = connectToReplica(replicaId, secretKeyFactory); + logger.debug("ClientID {}, updating connection to replica {}, at address: {}", clientId, + replicaId, controller.getRemoteAddress(replicaId)); + + future.awaitUninterruptibly(); + + if (!future.isSuccess()) { + logger.error("Impossible to connect to " + replicaId); + } + + } catch (InvalidKeyException | InvalidKeySpecException ex) { + logger.error("Failed to initialize MAC engine", ex); + } + rl.writeLock().unlock(); + } else { + rl.readLock().unlock(); + } + } + } catch (NoSuchAlgorithmException ex) { + logger.error("Failed to initialzie secret key factory", ex); + } + } + + @Override + public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { + if (cause instanceof ClosedChannelException) { + logger.error("Connection with replica closed.", cause); + } else if (cause instanceof ConnectException) { + logger.error("Impossible to connect to replica.", cause); + } else if (cause instanceof IOException) { + logger.error("Replica disconnected. Connection reset by peer."); + } else { + logger.error("Replica disconnected.", cause); + } + } + + @Override + public void channelRead0(ChannelHandlerContext ctx, TOMMessage sm) throws Exception { + logger.debug("channelRead0(ChannelHandlerContext ctx, TOMMessage sm)."); + + if (closed) { + closeChannelAndEventLoop(ctx.channel()); + return; + } + trr.replyReceived(sm); + } + + @Override + public void channelActive(ChannelHandlerContext ctx) { + + if (closed) { + closeChannelAndEventLoop(ctx.channel()); + return; + } + logger.debug("Channel active"); + } + + public void reconnect(final ChannelHandlerContext ctx) { + + rl.writeLock().lock(); + + ArrayList sessions = new ArrayList( + sessionClientToReplica.values()); + for (NettyClientServerSession ncss : sessions) { + if (ncss.getChannel() == ctx.channel()) { + int replicaId = ncss.getReplicaId(); + try { + + if (controller.getRemoteAddress(replicaId) != null) { + + ChannelFuture future; + try { + future = connectToReplica(replicaId, secretKeyFactory); + } catch (InvalidKeyException | InvalidKeySpecException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + logger.info("ClientID {}, re-connection to replica {}, at address: {}", clientId, replicaId, + controller.getRemoteAddress(replicaId)); + + } else { + // This cleans an old server from the session table + removeClient(replicaId); + } + } catch (NoSuchAlgorithmException ex) { + logger.error("Failed to reconnect to replica", ex); + } + } + } + + rl.writeLock().unlock(); + } + + @Override + public void setReplyReceiver(ReplyReceiver trr) { + this.trr = trr; + } + + @Override + public void send(boolean sign, int[] targets, TOMMessage sm) { + + int quorum; + + Integer[] targetArray = Arrays.stream(targets).boxed().toArray(Integer[]::new); + Collections.shuffle(Arrays.asList(targetArray), new Random()); + + if (controller.getStaticConf().isBFT()) { + quorum = (int) Math.ceil((controller.getCurrentViewN() + controller.getCurrentViewF()) / 2) + 1; + } else { + quorum = (int) Math.ceil((controller.getCurrentViewN()) / 2) + 1; + } + + listener.waitForChannels(quorum); // wait for the previous transmission to complete + + logger.debug("Sending request from " + sm.getSender() + " with sequence number " + sm.getSequence() + " to " + + Arrays.toString(targetArray)); + + if (sm.serializedMessage == null) { + + // serialize message + DataOutputStream dos = null; + try { + ByteArrayOutputStream baos = new ByteArrayOutputStream(); + dos = new DataOutputStream(baos); + sm.wExternal(dos); + dos.flush(); + sm.serializedMessage = baos.toByteArray(); + } catch (IOException ex) { + logger.debug("Impossible to serialize message: " + sm); + } + } + + // Logger.println("Sending message with "+sm.serializedMessage.length+" bytes of + // content."); + + // produce signature + if (sign && sm.serializedMessageSignature == null) { + sm.serializedMessageSignature = signMessage(privKey, sm.serializedMessage); + } + + int sent = 0; + + for (int target : targetArray) { + // This is done to avoid a race condition with the writeAndFush method. Since + // the method is asynchronous, + // each iteration of this loop could overwrite the destination of the previous + // one + try { + sm = (TOMMessage) sm.clone(); + } catch (CloneNotSupportedException e) { + logger.error("Failed to clone TOMMessage", e); + continue; + } + + sm.destination = targets[target]; + + rl.readLock().lock(); + Channel channel = ((NettyClientServerSession) sessionClientToReplica.get(targets[target])).getChannel(); + rl.readLock().unlock(); + if (channel.isActive()) { + sm.signed = sign; + ChannelFuture f = channel.writeAndFlush(sm); + + f.addListener(listener); + + sent++; + } else { + logger.debug("Channel to " + targets[target] + " is not connected"); + } + } + + if (targets.length > controller.getCurrentViewF() && sent < controller.getCurrentViewF() + 1) { + // if less than f+1 servers are connected send an exception to the client + throw new RuntimeException("Impossible to connect to servers!"); + } + if (targets.length == 1 && sent == 0) + throw new RuntimeException("Server not connected"); + } + + public void sign(TOMMessage sm) { + // serialize message + DataOutputStream dos = null; + byte[] data = null; + try { + ByteArrayOutputStream baos = new ByteArrayOutputStream(); + dos = new DataOutputStream(baos); + sm.wExternal(dos); + dos.flush(); + data = baos.toByteArray(); + sm.serializedMessage = data; + } catch (IOException ex) { + + logger.error("Failed to sign TOMMessage", ex); + } + + // produce signature + byte[] signature = signMessage(privKey, data); + sm.serializedMessageSignature = signature; + } + + public byte[] signMessage(PrivateKey key, byte[] message) { + // long startTime = System.nanoTime(); + try { + if (signatureEngine == null) { + signatureEngine = TOMUtil.getSigEngine(); + } + byte[] result = null; + + signatureEngine.initSign(key); + signatureEngine.update(message); + result = signatureEngine.sign(); + + // st.store(System.nanoTime() - startTime); + return result; + } catch (Exception e) { + logger.error("Failed to sign message", e); + return null; + } + } + + @Override + public void close() { + this.closed = true; + // Iterator sessions = sessionClientToReplica.values().iterator(); + rl.readLock().lock(); + ArrayList sessions = new ArrayList<>(sessionClientToReplica.values()); + rl.readLock().unlock(); + for (NettyClientServerSession ncss : sessions) { + Channel c = ncss.getChannel(); + closeChannelAndEventLoop(c); + } + } + + private ChannelInitializer getChannelInitializer() throws NoSuchAlgorithmException { + + final NettyClientPipelineFactory nettyClientPipelineFactory = new NettyClientPipelineFactory(this, + sessionClientToReplica, controller, rl); + + ChannelInitializer channelInitializer = new ChannelInitializer() { + @Override + public void initChannel(SocketChannel ch) throws Exception { + ch.pipeline().addLast(nettyClientPipelineFactory.getDecoder()); + ch.pipeline().addLast(nettyClientPipelineFactory.getEncoder()); + ch.pipeline().addLast(nettyClientPipelineFactory.getHandler()); + + } + }; + return channelInitializer; + } + + @Override + public void channelUnregistered(final ChannelHandlerContext ctx) throws Exception { + scheduleReconnect(ctx, 10); + } + + @Override + public void channelInactive(final ChannelHandlerContext ctx) { + scheduleReconnect(ctx, 10); + } + + private void closeChannelAndEventLoop(Channel c) { + // once having an event in your handler (EchoServerHandler) + // Close the current channel + c.close(); + // Then close the parent channel (the one attached to the bind) + if (c.parent() != null) { + c.parent().close(); + } + workerGroup.shutdownGracefully(); + } + + private void scheduleReconnect(final ChannelHandlerContext ctx, int time) { + if (closed) { + closeChannelAndEventLoop(ctx.channel()); + return; + } + + final EventLoop loop = ctx.channel().eventLoop(); + loop.schedule(new Runnable() { + @Override + public void run() { + reconnect(ctx); + } + }, time, TimeUnit.SECONDS); + } + + private class SyncListener implements GenericFutureListener { + + private int remainingFutures; + + private final Lock futureLock; + private final Condition enoughCompleted; + + public SyncListener() { + + this.remainingFutures = 0; + + this.futureLock = new ReentrantLock(); + this.enoughCompleted = futureLock.newCondition(); + } + + @Override + public void operationComplete(ChannelFuture f) { + + this.futureLock.lock(); + + this.remainingFutures--; + + if (this.remainingFutures <= 0) { + + this.enoughCompleted.signalAll(); + } + + logger.debug(this.remainingFutures + " channel operations remaining to complete"); + + this.futureLock.unlock(); + + } + + public void waitForChannels(int n) { + + this.futureLock.lock(); + if (this.remainingFutures > 0) { + + logger.debug("There are still " + this.remainingFutures + + " channel operations pending, waiting to complete"); + + try { + this.enoughCompleted.await(1000, TimeUnit.MILLISECONDS); // timeout if a malicous replica refuses to + // acknowledge the operation as + // completed + } catch (InterruptedException ex) { + logger.error("Interruption while waiting on condition", ex); + } + + } + + logger.debug("All channel operations completed or timed out"); + + this.remainingFutures = n; + + this.futureLock.unlock(); + } + + } + + /** + * Tulio Ribeiro Connect to specific replica and returns the ChannelFuture. + * sessionClientToReplica is replaced with the new connection. Removed redundant + * code. + */ + public synchronized ChannelFuture connectToReplica(int replicaId, SecretKeyFactory fac) + throws NoSuchAlgorithmException, InvalidKeySpecException, InvalidKeyException { + + String str = this.clientId + ":" + replicaId; + PBEKeySpec spec = TOMUtil.generateKeySpec(str.toCharArray()); + SecretKey authKey = fac.generateSecret(spec); + + Bootstrap b = new Bootstrap(); + b.group(workerGroup); + b.channel(NioSocketChannel.class); + b.option(ChannelOption.SO_KEEPALIVE, true); + b.option(ChannelOption.TCP_NODELAY, true); + b.option(ChannelOption.SO_SNDBUF, tcpSendBufferSize); + b.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, connectionTimeoutMsec); + b.handler(getChannelInitializer()); + + ChannelFuture channelFuture = b.connect(controller.getRemoteAddress(replicaId)); + + NettyClientServerSession ncss = new NettyClientServerSession( + channelFuture.channel(), replicaId); + sessionClientToReplica.put(replicaId, ncss); + + return channelFuture; + } + + public synchronized void removeClient(int clientId) { + sessionClientToReplica.remove(clientId); + } } diff --git a/src/bftsmart/communication/client/netty/NettyClientServerCommunicationSystemServerSide.java b/src/bftsmart/communication/client/netty/NettyClientServerCommunicationSystemServerSide.java index 5f74a8cfa..f59125f5f 100644 --- a/src/bftsmart/communication/client/netty/NettyClientServerCommunicationSystemServerSide.java +++ b/src/bftsmart/communication/client/netty/NettyClientServerCommunicationSystemServerSide.java @@ -35,11 +35,13 @@ import java.net.InetSocketAddress; import java.nio.channels.ClosedChannelException; import java.security.NoSuchAlgorithmException; +import java.security.PrivateKey; import java.util.HashMap; import java.util.Iterator; import java.util.Map.Entry; import java.util.Set; import java.util.ArrayList; +import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.locks.ReentrantReadWriteLock; import javax.crypto.Mac; @@ -54,206 +56,236 @@ import bftsmart.tom.util.TOMUtil; import java.net.InetAddress; import java.net.UnknownHostException; +import java.util.logging.Level; /** * * @author Paulo */ @Sharable -public class NettyClientServerCommunicationSystemServerSide extends SimpleChannelInboundHandler implements CommunicationSystemServerSide { +public class NettyClientServerCommunicationSystemServerSide extends SimpleChannelInboundHandler + implements CommunicationSystemServerSide { + + private Logger logger = LoggerFactory.getLogger(this.getClass()); + + private RequestReceiver requestReceiver; + private ConcurrentHashMap sessionReplicaToClient; + private ReentrantReadWriteLock rl; + private ServerViewController controller; + private boolean closed = false; + private Channel mainChannel; + + // This locked seems to introduce a bottleneck and seems useless, but I cannot + // recall why I added it + // private ReentrantLock sendLock = new ReentrantLock(); + private NettyServerPipelineFactory serverPipelineFactory; + + /* Tulio Ribeiro */ + private static int tcpSendBufferSize = 8 * 1024 * 1024; + private static int bossThreads = 8; /* listens and accepts on server socket; workers handle r/w I/O */ + private static int connectionBacklog = 1024; /* pending connections boss thread will queue to accept */ + private static int connectionTimeoutMsec = 40000; /* (40 seconds) */ + private PrivateKey privKey; + /* Tulio Ribeiro */ - private Logger logger = LoggerFactory.getLogger(this.getClass()); - - private RequestReceiver requestReceiver; - private HashMap sessionTable; - private ReentrantReadWriteLock rl; - private ServerViewController controller; - private boolean closed = false; - private Channel mainChannel; - - // This locked seems to introduce a bottleneck and seems useless, but I cannot recall why I added it - //private ReentrantLock sendLock = new ReentrantLock(); - private NettyServerPipelineFactory serverPipelineFactory; - public NettyClientServerCommunicationSystemServerSide(ServerViewController controller) { try { this.controller = controller; - sessionTable = new HashMap(); + /* Tulio Ribeiro */ + privKey = controller.getStaticConf().getPrivateKey(); + + sessionReplicaToClient = new ConcurrentHashMap<>(); rl = new ReentrantReadWriteLock(); - //Configure the server. - Mac macDummy = TOMUtil.getMacFactory(); - - serverPipelineFactory = new NettyServerPipelineFactory(this, sessionTable, macDummy.getMacLength(), controller, rl, TOMUtil.getSignatureSize(controller)); - - EventLoopGroup bossGroup = new NioEventLoopGroup(); - - //If the numbers of workers are not specified by the configuration file, - //the event group is created with the default number of threads, which - //should be twice the number of cores available. - int nWorkers = this.controller.getStaticConf().getNumNettyWorkers(); - EventLoopGroup workerGroup = (nWorkers > 0 ? new NioEventLoopGroup(nWorkers) : new NioEventLoopGroup()); - - ServerBootstrap b = new ServerBootstrap(); - b.group(bossGroup, workerGroup) - .channel(NioServerSocketChannel.class) - .childHandler(new ChannelInitializer() { - @Override - public void initChannel(SocketChannel ch) throws Exception { - ch.pipeline().addLast(serverPipelineFactory.getDecoder()); - ch.pipeline().addLast(serverPipelineFactory.getEncoder()); - ch.pipeline().addLast(serverPipelineFactory.getHandler()); + // Configure the server. + + serverPipelineFactory = new NettyServerPipelineFactory(this, sessionReplicaToClient, controller, rl); + + EventLoopGroup bossGroup = new NioEventLoopGroup(bossThreads); + EventLoopGroup workerGroup = new NioEventLoopGroup(Runtime.getRuntime().availableProcessors()); + + ServerBootstrap b = new ServerBootstrap(); + b.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class) + .option(ChannelOption.SO_REUSEADDR, true).option(ChannelOption.SO_KEEPALIVE, true) + .option(ChannelOption.TCP_NODELAY, true).option(ChannelOption.SO_SNDBUF, tcpSendBufferSize) + .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, connectionTimeoutMsec) + .option(ChannelOption.SO_BACKLOG, connectionBacklog) + .childHandler(new ChannelInitializer() { + @Override + public void initChannel(SocketChannel ch) throws Exception { + ch.pipeline().addLast(serverPipelineFactory.getDecoder()); + ch.pipeline().addLast(serverPipelineFactory.getEncoder()); + ch.pipeline().addLast(serverPipelineFactory.getHandler()); + } + }).childOption(ChannelOption.SO_KEEPALIVE, true).childOption(ChannelOption.TCP_NODELAY, true); + String myAddress; + String confAddress = controller.getStaticConf().getRemoteAddress(controller.getStaticConf().getProcessId()) + .getAddress().getHostAddress(); + + if (InetAddress.getLoopbackAddress().getHostAddress().equals(confAddress)) { + + myAddress = InetAddress.getLoopbackAddress().getHostAddress(); + + } + + else if (controller.getStaticConf().getBindAddress().equals("")) { + + myAddress = InetAddress.getLocalHost().getHostAddress(); + + // If Netty binds to the loopback address, clients will not be able to connect + // to replicas. + // To solve that issue, we bind to the address supplied in config/hosts.config + // instead. + if (InetAddress.getLoopbackAddress().getHostAddress().equals(myAddress) + && !myAddress.equals(confAddress)) { + + myAddress = confAddress; } - }) .childOption(ChannelOption.SO_KEEPALIVE, true).childOption(ChannelOption.TCP_NODELAY, true); - - String myAddress; - String confAddress = - controller.getStaticConf().getRemoteAddress(controller.getStaticConf().getProcessId()).getAddress().getHostAddress(); - - if (InetAddress.getLoopbackAddress().getHostAddress().equals(confAddress)) { - - myAddress = InetAddress.getLoopbackAddress().getHostAddress(); - - } - - else if (controller.getStaticConf().getBindAddress().equals("")) { - - myAddress = InetAddress.getLocalHost().getHostAddress(); - - //If Netty binds to the loopback address, clients will not be able to connect to replicas. - //To solve that issue, we bind to the address supplied in config/hosts.config instead. - if (InetAddress.getLoopbackAddress().getHostAddress().equals(myAddress) && !myAddress.equals(confAddress)) { - - myAddress = confAddress; - } - - - } else { - - myAddress = controller.getStaticConf().getBindAddress(); - } - - int myPort = controller.getStaticConf().getPort(controller.getStaticConf().getProcessId()); - - ChannelFuture f = b.bind(new InetSocketAddress(myAddress, myPort)).sync(); + + } else { + + myAddress = controller.getStaticConf().getBindAddress(); + } + + int myPort = controller.getStaticConf().getPort(controller.getStaticConf().getProcessId()); + + ChannelFuture f = b.bind(new InetSocketAddress(myAddress, myPort)).sync(); logger.info("ID = " + controller.getStaticConf().getProcessId()); logger.info("N = " + controller.getCurrentViewN()); logger.info("F = " + controller.getCurrentViewF()); - logger.info("Port = " + controller.getStaticConf().getPort(controller.getStaticConf().getProcessId())); + logger.info("Port (client <-> server) = " + + controller.getStaticConf().getPort(controller.getStaticConf().getProcessId())); + logger.info("Port (server <-> server) = " + + controller.getStaticConf().getServerToServerPort(controller.getStaticConf().getProcessId())); logger.info("requestTimeout = " + controller.getStaticConf().getRequestTimeout()); logger.info("maxBatch = " + controller.getStaticConf().getMaxBatchSize()); - if (controller.getStaticConf().getUseMACs() == 1) logger.info("Using MACs"); - if(controller.getStaticConf().getUseSignatures() == 1) logger.info("Using Signatures"); - logger.info("Binded replica to IP address " + myAddress); - //******* EDUARDO END **************// - - mainChannel = f.channel(); - - } catch (NoSuchAlgorithmException | InterruptedException | UnknownHostException ex) { - logger.error("Failed to create Netty communication system",ex); + if (controller.getStaticConf().getUseSignatures()) + logger.info("Using Signatures"); + logger.info("Binded replica to IP address " + myAddress); + // ******* EDUARDO END **************// + + /* Tulio Ribeiro */ + // SSL/TLS + logger.info("SSL/TLS enabled, protocol version: {}", controller.getStaticConf().getSSLTLSProtocolVersion()); + + /* Tulio Ribeiro END */ + + mainChannel = f.channel(); + + } catch (InterruptedException | UnknownHostException ex) { + logger.error("Failed to create Netty communication system", ex); } } - private void closeChannelAndEventLoop(Channel c) { - c.flush(); - c.deregister(); - c.close(); - c.eventLoop().shutdownGracefully(); - } - - @Override - public void shutdown() { - - logger.info("Shutting down Netty system"); - - this.closed = true; - - closeChannelAndEventLoop(mainChannel); - - rl.readLock().lock(); - ArrayList sessions = new ArrayList<>(sessionTable.values()); - rl.readLock().unlock(); - for (NettyClientServerSession ncss : sessions) { - - closeChannelAndEventLoop(ncss.getChannel()); - - } - - logger.info("NettyClientServerCommunicationSystemServerSide is halting."); - - } - + private void closeChannelAndEventLoop(Channel c) { + c.flush(); + c.deregister(); + c.close(); + c.eventLoop().shutdownGracefully(); + } + @Override - public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause){ - - if (this.closed) { - closeChannelAndEventLoop(ctx.channel()); - return; - } - - if(cause instanceof ClosedChannelException) - logger.info("Connection with client closed."); - else { - logger.error("Impossible to connect to client.",cause); + public void shutdown() { + + logger.info("Shutting down Netty system"); + + this.closed = true; + + closeChannelAndEventLoop(mainChannel); + + rl.readLock().lock(); + ArrayList sessions = new ArrayList<>(sessionReplicaToClient.values()); + rl.readLock().unlock(); + for (NettyClientServerSession ncss : sessions) { + + closeChannelAndEventLoop(ncss.getChannel()); + + } + + logger.info("NettyClientServerCommunicationSystemServerSide is halting."); + + } + + @Override + public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) { + + if (this.closed) { + closeChannelAndEventLoop(ctx.channel()); + return; + } + + if (cause instanceof ClosedChannelException) + logger.info("Client connection closed."); + else if (cause instanceof IOException) { + logger.error("Impossible to connect to client. (Connection reset by peer)"); + } else { + logger.error("Connection problem. Cause:{}", cause); } } @Override protected void channelRead0(ChannelHandlerContext ctx, TOMMessage sm) throws Exception { - if (this.closed) { - closeChannelAndEventLoop(ctx.channel()); - return; - } - - //delivers message to TOMLayer + if (this.closed) { + closeChannelAndEventLoop(ctx.channel()); + return; + } + + // delivers message to TOMLayer if (requestReceiver == null) logger.warn("Request receiver is still null!"); - else requestReceiver.requestReceived(sm); + else + requestReceiver.requestReceived(sm); } @Override public void channelActive(ChannelHandlerContext ctx) { - - if (this.closed) { - closeChannelAndEventLoop(ctx.channel()); - return; - } - logger.info("Session Created, active clients=" + sessionTable.size()); + + if (this.closed) { + closeChannelAndEventLoop(ctx.channel()); + return; + } + logger.info("Session Created, active clients=" + sessionReplicaToClient.size()); } @Override public void channelInactive(ChannelHandlerContext ctx) { - - if (this.closed) { - closeChannelAndEventLoop(ctx.channel()); - return; - } - - rl.writeLock().lock(); - try { - Set s = sessionTable.entrySet(); - Iterator i = s.iterator(); - while (i.hasNext()) { - Entry m = (Entry) i.next(); - NettyClientServerSession value = (NettyClientServerSession) m.getValue(); - if (ctx.channel().equals(value.getChannel())) { - int key = (Integer) m.getKey(); - logger.info("Removing client channel with ID= " + key); - sessionTable.remove(key); - logger.info("Active clients=" + sessionTable.size()); - break; - } + logger.debug("Channel Inactive"); + if (this.closed) { + closeChannelAndEventLoop(ctx.channel()); + return; + } + + // debugSessions(); + + Set s = sessionReplicaToClient.entrySet(); + Iterator i = s.iterator(); + while (i.hasNext()) { + Entry m = (Entry) i.next(); + NettyClientServerSession value = (NettyClientServerSession) m.getValue(); + if (ctx.channel().equals(value.getChannel())) { + int key = (Integer) m.getKey(); + toRemove(key); + break; } - - + } + + logger.debug("Session Closed, active clients=" + sessionReplicaToClient.size()); + } + + public synchronized void toRemove(Integer key) { - } finally { - rl.writeLock().unlock(); + Iterator it = sessionReplicaToClient.keySet().iterator(); + while (it.hasNext()) { + Integer cli = (Integer) it.next(); + logger.debug("SessionReplicaToClient: Key:{}, Value:{}", cli, sessionReplicaToClient.get(cli)); } - logger.debug("Session Closed, active clients=" + sessionTable.size()); + + logger.debug("Removing client channel with ID = " + key); + sessionReplicaToClient.remove(key); + } @Override @@ -264,7 +296,7 @@ public void setRequestReceiver(RequestReceiver tl) { @Override public void send(int[] targets, TOMMessage sm, boolean serializeClassHeaders) { - //serialize message + // serialize message DataOutputStream dos = null; byte[] data = null; @@ -279,98 +311,56 @@ public void send(int[] targets, TOMMessage sm, boolean serializeClassHeaders) { logger.error("Failed to serialize message.", ex); } - //replies are not signed in the current JBP version + // replies are not signed in the current JBP version sm.signed = false; - //produce signature if necessary (never in the current version) + // produce signature if necessary (never in the current version) if (sm.signed) { - //******* EDUARDO BEGIN **************// - byte[] data2 = TOMUtil.signMessage(controller.getStaticConf().getPrivateKey(), data); - //******* EDUARDO END **************// - sm.serializedMessageSignature = data2; + byte[] signature = TOMUtil.signMessage(privKey, data); + sm.serializedMessageSignature = signature; } - for (int i = 0; i < targets.length; i++) { + for (int target : targets) { + try { + sm = (TOMMessage) sm.clone(); + } catch (CloneNotSupportedException ex) { + logger.error("Failed to clone TOMMessage", ex); + continue; + } + rl.readLock().lock(); - //sendLock.lock(); - try { - NettyClientServerSession ncss = (NettyClientServerSession) sessionTable.get(targets[i]); - if (ncss != null) { - Channel session = ncss.getChannel(); - sm.destination = targets[i]; - //send message - session.writeAndFlush(sm); // This used to invoke "await". Removed to avoid blockage and race condition. - - ///////TODO: replace this patch for a proper client preamble - } else if (sm.getSequence() >= 0 && sm.getSequence() <= 5) { - - final int id = targets[i]; - final TOMMessage msg = sm; - - Thread t = new Thread() { - - public void run() { - - logger.warn("Received request from " + id + " before establishing Netty connection. Re-trying until connection is established"); - - NettyClientServerSession ncss = null; - while (ncss == null) { - - rl.readLock().lock(); - - try { - Thread.sleep(1000); - } catch (InterruptedException ex) { - logger.error("Interruption while sleeping", ex); - } - - ncss = (NettyClientServerSession) sessionTable.get(id); - if (ncss != null) { - Channel session = ncss.getChannel(); - msg.destination = id; - //send message - session.writeAndFlush(msg); - } - - rl.readLock().unlock(); - - } - - logger.info("Connection with " + id + " established!"); - - - } - - }; - - t.start(); - /////////////////////////////////////////// - } else { - logger.warn("!!!!!!!!NettyClientServerSession is NULL !!!!!! sequence: " + sm.getSequence() + ", ID; " + targets[i]); - } - } finally { - //sendLock.unlock(); - rl.readLock().unlock(); + if (sessionReplicaToClient.containsKey(target)) { + sm.destination = target; + sessionReplicaToClient.get(target).getChannel().writeAndFlush(sm); + } else { + logger.debug("Client not into sessionReplicaToClient({}):{}, waiting and retrying.", target, + sessionReplicaToClient.containsKey(target)); + /* + * ClientSession clientSession = new ClientSession(target, sm); new + * Thread(clientSession).start(); + */ + // should I wait for the client? } + rl.readLock().unlock(); } } - @Override - public int[] getClients() { - - rl.readLock().lock(); - Set s = sessionTable.keySet(); - int[] clients = new int[s.size()]; - Iterator it = s.iterator(); - int i = 0; - while (it.hasNext()) { - - clients[i] = ((Integer) it.next()).intValue(); - i++; - } - - rl.readLock().unlock(); - - return clients; - } + @Override + public int[] getClients() { + + rl.readLock().lock(); + Set s = sessionReplicaToClient.keySet(); + int[] clients = new int[s.size()]; + Iterator it = s.iterator(); + int i = 0; + while (it.hasNext()) { + + clients[i] = ((Integer) it.next()).intValue(); + i++; + } + + rl.readLock().unlock(); + + return clients; + } } diff --git a/src/bftsmart/communication/client/netty/NettyClientServerSession.java b/src/bftsmart/communication/client/netty/NettyClientServerSession.java index 66a2e31b2..e269b41c7 100644 --- a/src/bftsmart/communication/client/netty/NettyClientServerSession.java +++ b/src/bftsmart/communication/client/netty/NettyClientServerSession.java @@ -29,32 +29,17 @@ */ public class NettyClientServerSession { private Channel channel; - private Mac macSend; - private Mac macReceive; private int replicaId; private Lock lock; private int lastMsgReceived; - public NettyClientServerSession(Channel channel, Mac macSend, Mac macReceive, int replicaId) { + public NettyClientServerSession(Channel channel, int replicaId) { this.channel = channel; - this.macSend = macSend; - this.macReceive = macReceive; this.replicaId = replicaId; this.lock = new ReentrantLock(); this.lastMsgReceived = -1; } - - public Mac getMacReceive() { - return macReceive; - } - - - public Mac getMacSend() { - return macSend; - } - - public Channel getChannel() { return channel; } diff --git a/src/bftsmart/communication/client/netty/NettyServerPipelineFactory.java b/src/bftsmart/communication/client/netty/NettyServerPipelineFactory.java index a4ddf0910..2320a7816 100644 --- a/src/bftsmart/communication/client/netty/NettyServerPipelineFactory.java +++ b/src/bftsmart/communication/client/netty/NettyServerPipelineFactory.java @@ -20,6 +20,7 @@ import io.netty.handler.codec.MessageToByteEncoder; import java.util.HashMap; +import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.locks.ReentrantReadWriteLock; import bftsmart.reconfiguration.ServerViewController; @@ -27,27 +28,23 @@ public class NettyServerPipelineFactory{ NettyClientServerCommunicationSystemServerSide ncs; - HashMap sessionTable; - int macLength; - int signatureLength; + ConcurrentHashMap sessionTable; ServerViewController controller; ReentrantReadWriteLock rl; - public NettyServerPipelineFactory(NettyClientServerCommunicationSystemServerSide ncs, HashMap sessionTable, int macLength, ServerViewController controller, ReentrantReadWriteLock rl, int signatureLength) { + public NettyServerPipelineFactory(NettyClientServerCommunicationSystemServerSide ncs, ConcurrentHashMap sessionTable, ServerViewController controller, ReentrantReadWriteLock rl) { this.ncs = ncs; this.sessionTable = sessionTable; - this.macLength = macLength; - this.signatureLength = signatureLength; this.controller = controller; this.rl = rl; } public ByteToMessageDecoder getDecoder(){ - return new NettyTOMMessageDecoder(false, sessionTable,macLength,controller,rl,signatureLength,controller.getStaticConf().getUseMACs()==1?true:false); + return new NettyTOMMessageDecoder(false, sessionTable,controller,rl); } public MessageToByteEncoder getEncoder(){ - return new NettyTOMMessageEncoder(false, sessionTable, macLength,rl,signatureLength, controller.getStaticConf().getUseMACs()==1?true:false); + return new NettyTOMMessageEncoder(false, sessionTable,rl); } public SimpleChannelInboundHandler getHandler(){ diff --git a/src/bftsmart/communication/client/netty/NettyTOMMessageDecoder.java b/src/bftsmart/communication/client/netty/NettyTOMMessageDecoder.java index 8feb89608..16c042c4d 100644 --- a/src/bftsmart/communication/client/netty/NettyTOMMessageDecoder.java +++ b/src/bftsmart/communication/client/netty/NettyTOMMessageDecoder.java @@ -15,28 +15,20 @@ */ package bftsmart.communication.client.netty; -import io.netty.buffer.ByteBuf; -import io.netty.channel.ChannelHandlerContext; -import io.netty.handler.codec.ByteToMessageDecoder; - import java.io.ByteArrayInputStream; import java.io.DataInputStream; -import java.util.Arrays; import java.util.List; -import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.locks.ReentrantReadWriteLock; -import javax.crypto.Mac; -import javax.crypto.SecretKey; -import javax.crypto.SecretKeyFactory; -import javax.crypto.spec.PBEKeySpec; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import bftsmart.reconfiguration.ViewController; import bftsmart.tom.core.messages.TOMMessage; -import bftsmart.tom.util.TOMUtil; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; +import io.netty.buffer.ByteBuf; +import io.netty.channel.ChannelHandlerContext; +import io.netty.handler.codec.ByteToMessageDecoder; /** * @@ -52,44 +44,40 @@ public class NettyTOMMessageDecoder extends ByteToMessageDecoder { */ //private final int BENCHMARK_PERIOD = 10000; private boolean isClient; - private Map sessionTable; - //private Storage st; - private int macSize; - private int signatureSize; + private ConcurrentHashMap sessionTable; private ViewController controller; private boolean firstTime; private ReentrantReadWriteLock rl; - //******* EDUARDO BEGIN: commented out some unused variables **************// - //private long numReceivedMsgs = 0; - //private long lastMeasurementStart = 0; - //private long max=0; - //private Storage st; - //private int count = 0; - - //private Signature signatureEngine; - //******* EDUARDO END **************// - - private boolean useMAC; - - public NettyTOMMessageDecoder(boolean isClient, Map sessionTable, int macLength, ViewController controller, ReentrantReadWriteLock rl, int signatureLength, boolean useMAC) { + public NettyTOMMessageDecoder(boolean isClient, + ConcurrentHashMap sessionTable, + ViewController controller, + ReentrantReadWriteLock rl) { this.isClient = isClient; this.sessionTable = sessionTable; - this.macSize = macLength; this.controller = controller; this.firstTime = true; this.rl = rl; - this.signatureSize = signatureLength; - this.useMAC = useMAC; logger.debug("new NettyTOMMessageDecoder!!, isClient=" + isClient); + logger.trace("\n\t isClient: {};" + + "\n\t sessionTable: {};" + + "\n\t controller: {};" + + "\n\t firstTime: {};" + + "\n\t rl: {};" + + "\n\t signatureSize: {};", + new Object[] {isClient, + sessionTable.toString(), + controller, + firstTime, + rl}); } @Override protected void decode(ChannelHandlerContext context, ByteBuf buffer, List list) throws Exception { // Wait until the length prefix is available. - if (buffer.readableBytes() < 4) { + if (buffer.readableBytes() < Integer.BYTES) { return; } @@ -98,39 +86,22 @@ protected void decode(ChannelHandlerContext context, ByteBuf buffer, List 0) { + signature = new byte[size]; buffer.readBytes(signature); } @@ -144,60 +115,27 @@ protected void decode(ChannelHandlerContext context, ByteBuf buffer, List { - private Logger logger = LoggerFactory.getLogger(this.getClass()); - private boolean isClient; - private Map sessionTable; - private int macLength; - private int signatureLength; + private ConcurrentHashMap sessionTable; private ReentrantReadWriteLock rl; - private boolean useMAC; + - public NettyTOMMessageEncoder(boolean isClient, Map sessionTable, int macLength, ReentrantReadWriteLock rl, int signatureLength, boolean useMAC){ + public NettyTOMMessageEncoder(boolean isClient, + ConcurrentHashMap sessionTable, + ReentrantReadWriteLock rl){ this.isClient = isClient; this.sessionTable = sessionTable; - this.macLength = macLength; this.rl = rl; - this.signatureLength = signatureLength; - this.useMAC = useMAC; } @Override protected void encode(ChannelHandlerContext context, TOMMessage sm, ByteBuf buffer) throws Exception { byte[] msgData; - byte[] macData = null; byte[] signatureData = null; msgData = sm.serializedMessage; if (sm.signed){ //signature was already produced before signatureData = sm.serializedMessageSignature; - if (signatureData.length != signatureLength) - logger.warn("Message signature has size "+signatureData.length+" and should have "+signatureLength); } - if (useMAC) { - macData = produceMAC(sm.destination, msgData, sm.getSender()); - if(macData == null) { - logger.warn("Uses MAC and the MAC returned is null. Won't write to channel"); - return; - } - } - - int dataLength = 1+msgData.length+(macData==null?0:macData.length)+ - (signatureData==null?0:signatureData.length); - - //Logger.println("Sending message with "+dataLength+" bytes."); + int dataLength = Integer.BYTES + msgData.length + + Integer.BYTES + (signatureData != null ? signatureData.length : 0); + /* msg size */ buffer.writeInt(dataLength); - /* control byte indicating if the message is signed or not */ - buffer.writeByte(sm.signed==true?(byte)1:(byte)0); + /* data to be sent */ + buffer.writeInt(msgData.length); buffer.writeBytes(msgData); - /* MAC */ - if (useMAC) - buffer.writeBytes(macData); + /* signature */ - if (signatureData != null) - buffer.writeBytes(signatureData); - - context.flush(); - } - - byte[] produceMAC(int id, byte[] data, int me) { - NettyClientServerSession session = (NettyClientServerSession)sessionTable.get(id); - if(session == null) { - logger.warn("Session for client " + id + " is null"); - return null; + if (signatureData != null) { + buffer.writeInt(signatureData.length); + buffer.writeBytes(signatureData); + } else { + buffer.writeInt(0); } - Mac macSend = session.getMacSend(); - return macSend.doFinal(data); + + context.flush(); } } diff --git a/src/bftsmart/communication/server/ServerConnection.java b/src/bftsmart/communication/server/ServerConnection.java index edc8d7cda..8d1668a0a 100644 --- a/src/bftsmart/communication/server/ServerConnection.java +++ b/src/bftsmart/communication/server/ServerConnection.java @@ -18,10 +18,16 @@ import java.io.ByteArrayInputStream; import java.io.DataInputStream; import java.io.DataOutputStream; +import java.io.FileInputStream; +import java.io.FileNotFoundException; import java.io.IOException; import java.io.ObjectInputStream; import java.net.Socket; +import java.net.SocketException; import java.net.UnknownHostException; +import java.security.KeyManagementException; +import java.security.KeyStore; +import java.security.KeyStoreException; import java.security.NoSuchAlgorithmException; import java.util.Arrays; import java.util.concurrent.LinkedBlockingQueue; @@ -33,6 +39,13 @@ import javax.crypto.SecretKey; import javax.crypto.SecretKeyFactory; import javax.crypto.spec.PBEKeySpec; +import javax.net.ssl.HandshakeCompletedEvent; +import javax.net.ssl.HandshakeCompletedListener; +import javax.net.ssl.KeyManagerFactory; +import javax.net.ssl.SSLContext; +import javax.net.ssl.SSLSocket; +import javax.net.ssl.SSLSocketFactory; +import javax.net.ssl.TrustManagerFactory; import bftsmart.communication.SystemMessage; import bftsmart.reconfiguration.ServerViewController; @@ -42,6 +55,11 @@ import java.math.BigInteger; import java.security.PrivateKey; import java.security.PublicKey; +import java.security.SecureRandom; +import java.security.Security; +import java.security.UnrecoverableKeyException; +import java.security.cert.CertificateException; +import java.security.spec.InvalidKeySpecException; import java.util.HashSet; import org.slf4j.Logger; @@ -59,30 +77,39 @@ public class ServerConnection { private Logger logger = LoggerFactory.getLogger(this.getClass()); - //public static final String MAC_ALGORITHM = "HmacMD5"; private static final long POOL_TIME = 5000; - //private static final int SEND_QUEUE_SIZE = 50; private ServerViewController controller; - private Socket socket; + private SSLSocket socket; private DataOutputStream socketOutStream = null; private DataInputStream socketInStream = null; private int remoteId; private boolean useSenderThread; protected LinkedBlockingQueue outQueue;// = new LinkedBlockingQueue(SEND_QUEUE_SIZE); - private HashSet noMACs = null; // this is used to keep track of data to be sent without a MAC. - // It uses the reference id for that same data private LinkedBlockingQueue inQueue; - private SecretKey authKey = null; - private Mac macSend; - private Mac macReceive; - private int macSize; + private Lock connectLock = new ReentrantLock(); /** Only used when there is no sender Thread */ private Lock sendLock; private boolean doWork = true; + + private SecretKey secretKey = null; - public ServerConnection(ServerViewController controller, Socket socket, int remoteId, - LinkedBlockingQueue inQueue, ServiceReplica replica) { + /** + * Tulio A. Ribeiro + * TLS vars. + */ + private KeyManagerFactory kmf; + private KeyStore ks = null; + private FileInputStream fis = null; + private TrustManagerFactory trustMgrFactory; + private SSLContext context; + private SSLSocketFactory socketFactory; + private static final String SECRET = "MySeCreT_2hMOygBwY"; + + public ServerConnection(ServerViewController controller, + SSLSocket socket, int remoteId, + LinkedBlockingQueue inQueue, + ServiceReplica replica) { this.controller = controller; @@ -94,32 +121,19 @@ public ServerConnection(ServerViewController controller, Socket socket, int remo this.outQueue = new LinkedBlockingQueue(this.controller.getStaticConf().getOutQueueSize()); - this.noMACs = new HashSet(); // Connect to the remote process or just wait for the connection? - if (isToConnect()) { - //I have to connect to the remote server - try { - this.socket = new Socket(this.controller.getStaticConf().getHost(remoteId), - this.controller.getStaticConf().getServerToServerPort(remoteId)); - ServersCommunicationLayer.setSocketOptions(this.socket); - new DataOutputStream(this.socket.getOutputStream()).writeInt(this.controller.getStaticConf().getProcessId()); - - } catch (UnknownHostException ex) { - logger.error("Failed to connect to replica",ex); - } catch (IOException ex) { - logger.error("Failed to connect to replica",ex); - } - } - //else I have to wait a connection from the remote server - - if (this.socket != null) { - try { - socketOutStream = new DataOutputStream(this.socket.getOutputStream()); - socketInStream = new DataInputStream(this.socket.getInputStream()); - } catch (IOException ex) { - logger.error("Error creating connection to "+remoteId,ex); - } - } + if (isToConnect()) { + ssltlsCreateConnection(); + } + + if (this.socket != null) { + try { + socketOutStream = new DataOutputStream(this.socket.getOutputStream()); + socketInStream = new DataInputStream(this.socket.getInputStream()); + } catch (IOException ex) { + logger.error("Error creating connection to " + remoteId, ex); + } + } //******* EDUARDO BEGIN **************// this.useSenderThread = this.controller.getStaticConf().isUseSenderThread(); @@ -129,7 +143,6 @@ public ServerConnection(ServerViewController controller, Socket socket, int remo } else { sendLock = new ReentrantLock(); } - authenticateAndEstablishAuthKey(); if (!this.controller.getStaticConf().isTheTTP()) { if (this.controller.getStaticConf().getTTPId() == remoteId) { @@ -141,10 +154,28 @@ public ServerConnection(ServerViewController controller, Socket socket, int remo } //******* EDUARDO END **************// } - +/** + * Tulio A. Ribeiro. + * @return SecretKey + */ public SecretKey getSecretKey() { - return authKey; - } + if (secretKey != null) + return secretKey; + else { + SecretKeyFactory fac; + PBEKeySpec spec; + try { + fac = TOMUtil.getSecretFactory(); + spec = TOMUtil.generateKeySpec(SECRET.toCharArray()); + secretKey = fac.generateSecret(spec); + } catch (NoSuchAlgorithmException e) { + e.printStackTrace(); + } catch (InvalidKeySpecException e) { + e.printStackTrace(); + } + } + return secretKey; + } /** * Stop message sending and reception. @@ -159,63 +190,53 @@ public void shutdown() { /** * Used to send packets to the remote server. */ - public final void send(byte[] data, boolean useMAC) throws InterruptedException { - if (useSenderThread) { - //only enqueue messages if there queue is not full - if (!useMAC) { - logger.debug("Not sending defaultMAC " + System.identityHashCode(data)); - noMACs.add(System.identityHashCode(data)); - } - - if (!outQueue.offer(data)) { - logger.debug("Out queue for " + remoteId + " full (message discarded)."); - } - } else { - sendLock.lock(); - sendBytes(data, useMAC); - sendLock.unlock(); - } + public final void send(byte[] data) throws InterruptedException { + if (useSenderThread) { + // only enqueue messages if there queue is not full + if (!outQueue.offer(data)) { + logger.debug("Out queue for " + remoteId + " full (message discarded)."); + } + } else { + sendLock.lock(); + sendBytes(data); + sendLock.unlock(); + } } /** - * try to send a message through the socket - * if some problem is detected, a reconnection is done - */ - private final void sendBytes(byte[] messageData, boolean useMAC) { - boolean abort = false; - do { - if (abort) return; // if there is a need to reconnect, abort this method - if (socket != null && socketOutStream != null) { - try { - //do an extra copy of the data to be sent, but on a single out stream write - byte[] mac = (useMAC && this.controller.getStaticConf().getUseMACs() == 1)?macSend.doFinal(messageData):null; - byte[] data = new byte[5 +messageData.length+((mac!=null)?mac.length:0)]; - int value = messageData.length; - - System.arraycopy(new byte[]{(byte)(value >>> 24),(byte)(value >>> 16),(byte)(value >>> 8),(byte)value},0,data,0,4); - System.arraycopy(messageData,0,data,4,messageData.length); - if(mac != null) { - //System.arraycopy(mac,0,data,4+messageData.length,mac.length); - System.arraycopy(new byte[]{ (byte) 1},0,data,4+messageData.length,1); - System.arraycopy(mac,0,data,5+messageData.length,mac.length); - } else { - System.arraycopy(new byte[]{(byte) 0},0,data,4+messageData.length,1); - } - - socketOutStream.write(data); - - return; - } catch (IOException ex) { - closeSocket(); - waitAndConnect(); - abort = true; - } - } else { - waitAndConnect(); - abort = true; - } - } while (doWork); - } + * try to send a message through the socket if some problem is detected, a + * reconnection is done + */ + private final void sendBytes(byte[] messageData) { + boolean abort = false; + do { + if (abort) + return; // if there is a need to reconnect, abort this method + if (socket != null && socketOutStream != null) { + try { + // do an extra copy of the data to be sent, but on a single out stream write + byte[] data = new byte[5 + messageData.length];// without MAC + int value = messageData.length; + + System.arraycopy(new byte[] { (byte) (value >>> 24), (byte) (value >>> 16), (byte) (value >>> 8), + (byte) value }, 0, data, 0, 4); + System.arraycopy(messageData, 0, data, 4, messageData.length); + System.arraycopy(new byte[] { (byte) 0 }, 0, data, 4 + messageData.length, 1); + + socketOutStream.write(data); + + return; + } catch (IOException ex) { + closeSocket(); + waitAndConnect(); + abort = true; + } + } else { + waitAndConnect(); + abort = true; + } + } while (doWork); + } //******* EDUARDO BEGIN **************// //return true of a process shall connect to the remote process, false otherwise @@ -273,143 +294,39 @@ from that same code (its the only part I understand why is necessary) * @param newSocket socket created when this server accepted the connection * (only used if processId is less than remoteId) */ - protected void reconnect(Socket newSocket) { - - connectLock.lock(); - - if (socket == null || !socket.isConnected()) { + protected void reconnect(SSLSocket newSocket) { - try { - - //******* EDUARDO BEGIN **************// - if (isToConnect()) { + connectLock.lock(); - socket = new Socket(this.controller.getStaticConf().getHost(remoteId), - this.controller.getStaticConf().getServerToServerPort(remoteId)); - ServersCommunicationLayer.setSocketOptions(socket); - new DataOutputStream(socket.getOutputStream()).writeInt(this.controller.getStaticConf().getProcessId()); + if (socket == null || !socket.isConnected()) { - //******* EDUARDO END **************// - } else { - socket = newSocket; - } - } catch (UnknownHostException ex) { - logger.error("Failed to connect to replica",ex); - } catch (IOException ex) { - - logger.error("Impossible to reconnect to replica " + remoteId + ": " + ex.getMessage()); - //ex.printStackTrace(); - } + if (isToConnect()) { + ssltlsCreateConnection(); + } else { + socket = newSocket; + } - if (socket != null) { - try { - socketOutStream = new DataOutputStream(socket.getOutputStream()); - socketInStream = new DataInputStream(socket.getInputStream()); - - authKey = null; - authenticateAndEstablishAuthKey(); - } catch (IOException ex) { - logger.error("Failed to authenticate to replica",ex); - } - } - } + if (socket != null) { + try { + socketOutStream = new DataOutputStream(socket.getOutputStream()); + socketInStream = new DataInputStream(socket.getInputStream()); - connectLock.unlock(); - } + // authKey = null; + // authenticateAndEstablishAuthKey(); + } catch (IOException ex) { + logger.error("Failed to authenticate to replica", ex); + } + } + } - //TODO! - public void authenticateAndEstablishAuthKey() { - if (authKey != null || socketOutStream == null || socketInStream == null) { - return; - } - - try { - //if (conf.getProcessId() > remoteId) { - // I asked for the connection, so I'm first on the auth protocol - //DataOutputStream dos = new DataOutputStream(socket.getOutputStream()); - //} else { - // I received a connection request, so I'm second on the auth protocol - //DataInputStream dis = new DataInputStream(socket.getInputStream()); - //} - - //Derive DH private key from replica's own private key - - PrivateKey privKey = controller.getStaticConf().getPrivateKey(); - BigInteger DHPrivKey = - new BigInteger(privKey.getEncoded()); - - //Create DH public key - BigInteger myDHPubKey = - controller.getStaticConf().getDHG().modPow(DHPrivKey, controller.getStaticConf().getDHP()); - - //turn it into a byte array - byte[] bytes = myDHPubKey.toByteArray(); - - byte[] signature = TOMUtil.signMessage(privKey, bytes); - - //send my DH public key and signature - socketOutStream.writeInt(bytes.length); - socketOutStream.write(bytes); - - socketOutStream.writeInt(signature.length); - socketOutStream.write(signature); - - //receive remote DH public key and signature - int dataLength = socketInStream.readInt(); - bytes = new byte[dataLength]; - int read = 0; - do { - read += socketInStream.read(bytes, read, dataLength - read); - - } while (read < dataLength); - - byte[] remote_Bytes = bytes; - - dataLength = socketInStream.readInt(); - bytes = new byte[dataLength]; - read = 0; - do { - read += socketInStream.read(bytes, read, dataLength - read); - - } while (read < dataLength); - - byte[] remote_Signature = bytes; - - //verify signature - PublicKey remotePubkey = controller.getStaticConf().getPublicKey(remoteId); - - if (!TOMUtil.verifySignature(remotePubkey, remote_Bytes, remote_Signature)) { - - logger.warn(remoteId + " sent an invalid signature!"); - shutdown(); - return; - } - - BigInteger remoteDHPubKey = new BigInteger(remote_Bytes); - - //Create secret key - BigInteger secretKey = - remoteDHPubKey.modPow(DHPrivKey, controller.getStaticConf().getDHP()); - - logger.info("Diffie-Hellman complete with " + remoteId); - - SecretKeyFactory fac = TOMUtil.getSecretFactory(); - PBEKeySpec spec = new PBEKeySpec(secretKey.toString().toCharArray()); - - //PBEKeySpec spec = new PBEKeySpec(PASSWORD.toCharArray()); - authKey = fac.generateSecret(spec); - - macSend = TOMUtil.getMacFactory(); - macSend.init(authKey); - macReceive = TOMUtil.getMacFactory(); - macReceive.init(authKey); - macSize = macSend.getMacLength(); - } catch (Exception ex) { - logger.error("Failed to create secret key",ex); - } - } + connectLock.unlock(); + } + private void closeSocket() { + + connectLock.lock(); + if (socket != null) { try { socketOutStream.flush(); @@ -424,6 +341,8 @@ private void closeSocket() { socketOutStream = null; socketInStream = null; } + + connectLock.unlock(); } private void waitAndConnect() { @@ -459,12 +378,9 @@ public void run() { } if (data != null) { - //sendBytes(data, noMACs.contains(System.identityHashCode(data))); - int ref = System.identityHashCode(data); - boolean sendMAC = !noMACs.remove(ref); - logger.debug((sendMAC ? "Sending" : "Not sending") + " MAC for data " + ref); - sendBytes(data, sendMAC); - } + logger.trace("Sending data to, RemoteId:{}", remoteId); + sendBytes(data); + } } logger.debug("Sender for " + remoteId + " stopped!"); @@ -482,65 +398,51 @@ public ReceiverThread() { @Override public void run() { - byte[] receivedMac = null; - try { - receivedMac = new byte[TOMUtil.getMacFactory().getMacLength()]; - } catch (NoSuchAlgorithmException ex) { - logger.error("Failed to get MAC vector object",ex); - } - - while (doWork) { - if (socket != null && socketInStream != null) { - try { - //read data length - int dataLength = socketInStream.readInt(); - byte[] data = new byte[dataLength]; - - //read data - int read = 0; - do { - read += socketInStream.read(data, read, dataLength - read); - } while (read < dataLength); - - //read mac - boolean result = true; - - byte hasMAC = socketInStream.readByte(); - if (controller.getStaticConf().getUseMACs() == 1 && hasMAC == 1) { - read = 0; - do { - read += socketInStream.read(receivedMac, read, macSize - read); - } while (read < macSize); - - result = Arrays.equals(macReceive.doFinal(data), receivedMac); - } - - if (result) { - SystemMessage sm = (SystemMessage) (new ObjectInputStream(new ByteArrayInputStream(data)).readObject()); - sm.authenticated = (controller.getStaticConf().getUseMACs() == 1 && hasMAC == 1); - - if (sm.getSender() == remoteId) { - if (!inQueue.offer(sm)) { - logger.warn("Inqueue full (message from " + remoteId + " discarded)."); - } - } - } else { - //TODO: violation of authentication... we should do something - logger.warn("Violation of authentication in message received from " + remoteId); - } - } catch (ClassNotFoundException ex) { - //invalid message sent, just ignore; - } catch (IOException ex) { - if (doWork) { - logger.debug("Closing socket and reconnecting"); - closeSocket(); - waitAndConnect(); - } - } - } else { - waitAndConnect(); - } - } + + while (doWork) { + if (socket != null && socketInStream != null) { + + try { + // read data length + int dataLength = socketInStream.readInt(); + byte[] data = new byte[dataLength]; + + // read data + int read = 0; + do { + read += socketInStream.read(data, read, dataLength - read); + } while (read < dataLength); + + byte hasMAC = socketInStream.readByte(); + + logger.trace("Read: {}, HasMAC: {}", read, hasMAC); + + SystemMessage sm = (SystemMessage) (new ObjectInputStream(new ByteArrayInputStream(data)) + .readObject()); + + //The verification it is done for the SSL/TLS protocol. + sm.authenticated = true; + + if (sm.getSender() == remoteId) { + if (!inQueue.offer(sm)) { + logger.warn("Inqueue full (message from " + remoteId + " discarded)."); + }/* else { + logger.trace("Message: {} queued, remoteId: {}", sm.toString(), sm.getSender()); + }*/ + } + } catch (ClassNotFoundException ex) { + logger.info("Invalid message received. Ignoring!"); + } catch (IOException ex) { + if (doWork) { + logger.debug("Closing socket and reconnecting"); + closeSocket(); + waitAndConnect(); + } + } + } else { + waitAndConnect(); + } + } } } @@ -562,70 +464,147 @@ public TTPReceiverThread(ServiceReplica replica) { } @Override - public void run() { - byte[] receivedMac = null; - try { - receivedMac = new byte[TOMUtil.getMacFactory().getMacLength()]; - } catch (NoSuchAlgorithmException ex) { - } - - while (doWork) { - if (socket != null && socketInStream != null) { - try { - //read data length - int dataLength = socketInStream.readInt(); - - byte[] data = new byte[dataLength]; - - //read data - int read = 0; - do { - read += socketInStream.read(data, read, dataLength - read); - } while (read < dataLength); - - //read mac - boolean result = true; - - byte hasMAC = socketInStream.readByte(); - if (controller.getStaticConf().getUseMACs() == 1 && hasMAC == 1) { - - read = 0; - do { - read += socketInStream.read(receivedMac, read, macSize - read); - } while (read < macSize); - - result = Arrays.equals(macReceive.doFinal(data), receivedMac); - } - - if (result) { - SystemMessage sm = (SystemMessage) (new ObjectInputStream(new ByteArrayInputStream(data)).readObject()); - - if (sm.getSender() == remoteId) { - //System.out.println("Mensagem recebia de: "+remoteId); - /*if (!inQueue.offer(sm)) { - bftsmart.tom.util.Logger.println("(ReceiverThread.run) in queue full (message from " + remoteId + " discarded)."); - System.out.println("(ReceiverThread.run) in queue full (message from " + remoteId + " discarded)."); - }*/ - this.replica.joinMsgReceived((VMMessage) sm); - } - } else { - //TODO: violation of authentication... we should do something - logger.warn("Violation of authentication in message received from " + remoteId); - } - } catch (ClassNotFoundException ex) { - logger.error("Failed to deserialize message",ex); - } catch (IOException ex) { - //ex.printStackTrace(); - if (doWork) { - closeSocket(); - waitAndConnect(); - } - } - } else { - waitAndConnect(); - } - } - } + public void run() { + + while (doWork) { + if (socket != null && socketInStream != null) { + try { + // read data length + int dataLength = socketInStream.readInt(); + + byte[] data = new byte[dataLength]; + + // read data + int read = 0; + do { + read += socketInStream.read(data, read, dataLength - read); + } while (read < dataLength); + + SystemMessage sm = (SystemMessage) (new ObjectInputStream(new ByteArrayInputStream(data)) + .readObject()); + + if (sm.getSender() == remoteId) { + this.replica.joinMsgReceived((VMMessage) sm); + } + + } catch (ClassNotFoundException ex) { + logger.error("Failed to deserialize message", ex); + } catch (IOException ex) { + // ex.printStackTrace(); + if (doWork) { + closeSocket(); + waitAndConnect(); + } + } + } else { + waitAndConnect(); + } + } + } } //******* EDUARDO END **************// + + + /** + * Deal with the creation of SSL/TLS connection. + * Author: Tulio A. Ribeiro + * + * @throws KeyStoreException + * @throws IOException + * @throws CertificateException + * @throws NoSuchAlgorithmException + * @throws UnrecoverableKeyException + * @throws KeyManagementException + * @throws InvalidKeySpecException + */ + + public void ssltlsCreateConnection() { + + SecretKeyFactory fac; + PBEKeySpec spec; + try { + fac = TOMUtil.getSecretFactory(); + spec = TOMUtil.generateKeySpec(SECRET.toCharArray()); + secretKey = fac.generateSecret(spec); + } catch (NoSuchAlgorithmException e) { + e.printStackTrace(); + } catch (InvalidKeySpecException e) { + e.printStackTrace(); + } + + String algorithm = Security.getProperty("ssl.KeyManagerFactory.algorithm"); + try { + fis = new FileInputStream("config/keysSSL_TLS/" + this.controller.getStaticConf().getSSLTLSKeyStore()); + ks = KeyStore.getInstance(KeyStore.getDefaultType()); + ks.load(fis, SECRET.toCharArray()); + } catch (FileNotFoundException e) { + e.printStackTrace(); + } catch (KeyStoreException e) { + e.printStackTrace(); + } catch (NoSuchAlgorithmException e) { + e.printStackTrace(); + } catch (CertificateException e) { + e.printStackTrace(); + } catch (IOException e) { + e.printStackTrace(); + } finally { + if (fis != null) { + try { + fis.close(); + } catch (IOException e) { + e.printStackTrace(); + } + } + } + try { + kmf = KeyManagerFactory.getInstance(algorithm); + kmf.init(ks, SECRET.toCharArray()); + + trustMgrFactory = TrustManagerFactory.getInstance(algorithm); + trustMgrFactory.init(ks); + context = SSLContext.getInstance(this.controller.getStaticConf().getSSLTLSProtocolVersion()); + context.init(kmf.getKeyManagers(), trustMgrFactory.getTrustManagers(), new SecureRandom()); + socketFactory = context.getSocketFactory(); + + } catch (KeyStoreException e) { + e.printStackTrace(); + } catch (NoSuchAlgorithmException e) { + e.printStackTrace(); + } catch (UnrecoverableKeyException e) { + e.printStackTrace(); + } catch (KeyManagementException e) { + e.printStackTrace(); + } + // Create the connection. + try { + this.socket = (SSLSocket) socketFactory.createSocket(this.controller.getStaticConf().getHost(remoteId), + this.controller.getStaticConf().getServerToServerPort(remoteId)); + this.socket.setKeepAlive(true); + this.socket.setTcpNoDelay(true); + this.socket.setEnabledCipherSuites(this.controller.getStaticConf().getEnabledCiphers()); + + this.socket.addHandshakeCompletedListener(new HandshakeCompletedListener() { + @Override + public void handshakeCompleted(HandshakeCompletedEvent event) { + logger.info("SSL/TLS handshake complete!, Id:{}" + " ## CipherSuite: {}.", remoteId, + event.getCipherSuite()); + } + }); + + this.socket.startHandshake(); + + ServersCommunicationLayer.setSSLSocketOptions(this.socket); + new DataOutputStream(this.socket.getOutputStream()) + .writeInt(this.controller.getStaticConf().getProcessId()); + + } catch (SocketException e) { + logger.error("Connection refused (SocketException)"); + // e.printStackTrace(); + } catch (UnknownHostException e) { + e.printStackTrace(); + } catch (IOException e) { + e.printStackTrace(); + } + + } } diff --git a/src/bftsmart/communication/server/ServersCommunicationLayer.java b/src/bftsmart/communication/server/ServersCommunicationLayer.java index a80eebc45..b9d058c76 100644 --- a/src/bftsmart/communication/server/ServersCommunicationLayer.java +++ b/src/bftsmart/communication/server/ServersCommunicationLayer.java @@ -17,12 +17,16 @@ import java.io.ByteArrayOutputStream; import java.io.DataInputStream; +import java.io.FileInputStream; import java.io.IOException; import java.io.ObjectOutputStream; import java.net.ServerSocket; import java.net.Socket; import java.net.SocketException; import java.net.SocketTimeoutException; +import java.security.KeyStore; +import java.security.SecureRandom; +import java.security.Security; import java.util.Iterator; import java.util.LinkedList; import java.util.List; @@ -35,11 +39,21 @@ import bftsmart.tom.ServiceReplica; import bftsmart.tom.util.TOMUtil; import java.net.InetAddress; +import java.util.Arrays; +import java.util.Collections; import java.util.HashMap; +import java.util.Random; import javax.crypto.SecretKey; import javax.crypto.SecretKeyFactory; import javax.crypto.spec.PBEKeySpec; +import javax.net.ssl.KeyManagerFactory; +import javax.net.ssl.SSLContext; +import javax.net.ssl.SSLHandshakeException; +import javax.net.ssl.SSLServerSocket; +import javax.net.ssl.SSLServerSocketFactory; +import javax.net.ssl.SSLSocket; +import javax.net.ssl.TrustManagerFactory; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -48,6 +62,23 @@ * * @author alysson */ + +/** + * Tulio A. Ribeiro. + * + * Generate a KeyPair used by SSL/TLS connections. Note that keypass argument is + * equal to the variable SECRET. + * + * The command generates the secret key.*/ +//## Elliptic Curve + //$keytool -genkey -keyalg EC -alias bftsmartEC -keypass MySeCreT_2hMOygBwY -keystore ./ecKeyPair -dname "CN=BFT-SMaRT" + //$keytool -importkeystore -srckeystore ./ecKeyPair -destkeystore ./ecKeyPair -deststoretype pkcs12 + +//## RSA + //$keytool -genkey -keyalg RSA -keysize 2048 -alias bftsmartRSA -keypass MySeCreT_2hMOygBwY -keystore ./RSA_KeyPair_2048.pkcs12 -dname "CN=BFT-SMaRT" + //$keytool -importkeystore -srckeystore ./RSA_KeyPair_2048.pkcs12 -destkeystore ./RSA_KeyPair_2048.pkcs12 -deststoretype pkcs12 + + public class ServersCommunicationLayer extends Thread { private Logger logger = LoggerFactory.getLogger(this.getClass()); @@ -61,76 +92,115 @@ public class ServersCommunicationLayer extends Thread { private boolean doWork = true; private Lock connectionsLock = new ReentrantLock(); private ReentrantLock waitViewLock = new ReentrantLock(); - //private Condition canConnect = waitViewLock.newCondition(); private List pendingConn = new LinkedList(); private ServiceReplica replica; - private SecretKey selfPwd; - private static final String PASSWORD = "commsyst"; + + + /** + * Tulio A. Ribeiro + * SSL / TLS. + */ + + private KeyManagerFactory kmf; + private KeyStore ks; + private TrustManagerFactory trustMgrFactory; + private SSLContext context; + private SSLServerSocketFactory serverSocketFactory; + private static final String SECRET = "MySeCreT_2hMOygBwY"; + private SecretKey selfPwd; + private SSLServerSocket serverSocketSSLTLS; + private String ssltlsProtocolVersion; public ServersCommunicationLayer(ServerViewController controller, - LinkedBlockingQueue inQueue, ServiceReplica replica) throws Exception { + LinkedBlockingQueue inQueue, + ServiceReplica replica) throws Exception { this.controller = controller; this.inQueue = inQueue; this.me = controller.getStaticConf().getProcessId(); this.replica = replica; - - //Try connecting if a member of the current view. Otherwise, wait until the Join has been processed! - if (controller.isInCurrentView()) { - int[] initialV = controller.getCurrentViewAcceptors(); - for (int i = 0; i < initialV.length; i++) { - if (initialV[i] != me) { - getConnection(initialV[i]); - } - } - } + this.ssltlsProtocolVersion = controller.getStaticConf().getSSLTLSProtocolVersion(); String myAddress; String confAddress = controller.getStaticConf().getRemoteAddress(controller.getStaticConf().getProcessId()).getAddress().getHostAddress(); if (InetAddress.getLoopbackAddress().getHostAddress().equals(confAddress)) { - myAddress = InetAddress.getLoopbackAddress().getHostAddress(); - } - else if (controller.getStaticConf().getBindAddress().equals("")) { - myAddress = InetAddress.getLocalHost().getHostAddress(); - //If the replica binds to the loopback address, clients will not be able to connect to replicas. //To solve that issue, we bind to the address supplied in config/hosts.config instead. if (InetAddress.getLoopbackAddress().getHostAddress().equals(myAddress) && !myAddress.equals(confAddress)) { - myAddress = confAddress; } - - } else { - myAddress = controller.getStaticConf().getBindAddress(); } int myPort = controller.getStaticConf().getServerToServerPort(controller.getStaticConf().getProcessId()); - - serverSocket = new ServerSocket(myPort, 50, InetAddress.getByName(myAddress)); - - /*serverSocket = new ServerSocket(controller.getStaticConf().getServerToServerPort( - controller.getStaticConf().getProcessId()));*/ - - SecretKeyFactory fac = TOMUtil.getSecretFactory(); - PBEKeySpec spec = new PBEKeySpec(PASSWORD.toCharArray()); - selfPwd = fac.generateSecret(spec); - - serverSocket.setSoTimeout(10000); - serverSocket.setReuseAddress(true); + FileInputStream fis = null; + try { + fis = new FileInputStream("config/keysSSL_TLS/" + controller.getStaticConf().getSSLTLSKeyStore()); + ks = KeyStore.getInstance(KeyStore.getDefaultType()); + ks.load(fis, SECRET.toCharArray()); + } finally { + if (fis != null) { + fis.close(); + } + } + + String algorithm = Security.getProperty("ssl.KeyManagerFactory.algorithm"); + kmf = KeyManagerFactory.getInstance(algorithm); + kmf.init(ks, SECRET.toCharArray()); + + trustMgrFactory = TrustManagerFactory.getInstance(algorithm); + trustMgrFactory.init(ks); + + context = SSLContext.getInstance(this.ssltlsProtocolVersion); + context.init(kmf.getKeyManagers(), trustMgrFactory.getTrustManagers(), new SecureRandom()); + + serverSocketFactory = context.getServerSocketFactory(); + this.serverSocketSSLTLS = (SSLServerSocket) serverSocketFactory.createServerSocket(myPort, 100, + InetAddress.getByName(myAddress)); + + serverSocketSSLTLS.setEnabledCipherSuites(this.controller.getStaticConf().getEnabledCiphers()); + + String[] ciphers = serverSocketFactory.getSupportedCipherSuites(); + for (int i = 0; i < ciphers.length; i++) { + logger.trace("Supported Cipher: {} ", ciphers[i]); + } + + //serverSocketSSLTLS.setPerformancePreferences(0, 2, 1); + //serverSocketSSLTLS.setSoTimeout(connectionTimeoutMsec); + serverSocketSSLTLS.setEnableSessionCreation(true); + serverSocketSSLTLS.setReuseAddress(true); + serverSocketSSLTLS.setNeedClientAuth(true); + serverSocketSSLTLS.setWantClientAuth(true); + + + SecretKeyFactory fac = TOMUtil.getSecretFactory(); + PBEKeySpec spec = TOMUtil.generateKeySpec(SECRET.toCharArray()); + selfPwd = fac.generateSecret(spec); + + //Try connecting if a member of the current view. Otherwise, wait until the Join has been processed! + if (controller.isInCurrentView()) { + int[] initialV = controller.getCurrentViewAcceptors(); + for (int i = 0; i < initialV.length; i++) { + if (initialV[i] != me) { + getConnection(initialV[i]); + } + } + } + start(); } public SecretKey getSecretKey(int id) { - if (id == controller.getStaticConf().getProcessId()) return selfPwd; + if (id == controller.getStaticConf().getProcessId()) + return selfPwd; else return connections.get(id).getSecretKey(); } @@ -173,7 +243,8 @@ private ServerConnection getConnection(int remoteId) { connectionsLock.lock(); ServerConnection ret = this.connections.get(remoteId); if (ret == null) { - ret = new ServerConnection(controller, null, remoteId, this.inQueue, this.replica); + ret = new ServerConnection(controller, null, + remoteId, this.inQueue, this.replica); this.connections.put(remoteId, ret); } connectionsLock.unlock(); @@ -191,23 +262,28 @@ public final void send(int[] targets, SystemMessage sm, boolean useMAC) { } byte[] data = bOut.toByteArray(); - - for (int i : targets) { - try { - if (i == me) { - sm.authenticated = true; - inQueue.put(sm); - } else { - //System.out.println("Going to send message to: "+i); - //******* EDUARDO BEGIN **************// - //connections[i].send(data); - getConnection(i).send(data, useMAC); - //******* EDUARDO END **************// - } - } catch (InterruptedException ex) { - logger.error("Interruption while inserting message into inqueue",ex); - } - } + + // this shuffling is done to prevent the replica with the lowest ID/index from being always + // the last one receiving the messages, which can result in that replica to become consistently + // delayed in relation to the others. + /*Tulio A. Ribeiro*/ + Integer[] targetsShuffled = Arrays.stream( targets ).boxed().toArray( Integer[]::new ); + Collections.shuffle(Arrays.asList(targetsShuffled), new Random(System.nanoTime())); + + for (int target : targetsShuffled) { + try { + if (target == me) { + sm.authenticated = true; + inQueue.put(sm); + logger.debug("Queueing (delivering) my own message, me:{}", target); + } else { + logger.debug("Sending message from:{} -> to:{}.", me, target); + getConnection(target).send(data); + } + } catch (InterruptedException ex) { + logger.error("Interruption while inserting message into inqueue", ex); + } + } } public void shutdown() { @@ -220,9 +296,6 @@ public void shutdown() { int[] activeServers = controller.getCurrentViewAcceptors(); for (int i = 0; i < activeServers.length; i++) { - //if (connections[i] != null) { - // connections[i].shutdown(); - //} if (me != activeServers[i]) { getConnection(activeServers[i]).shutdown(); } @@ -254,9 +327,9 @@ public void run() { //System.out.println("Waiting for server connections"); - Socket newSocket = serverSocket.accept(); + SSLSocket newSocket = (SSLSocket) serverSocketSSLTLS.accept(); + setSSLSocketOptions(newSocket); - ServersCommunicationLayer.setSocketOptions(newSocket); int remoteId = new DataInputStream(newSocket.getInputStream()).readInt(); //******* EDUARDO BEGIN **************// @@ -266,16 +339,18 @@ public void run() { pendingConn.add(new PendingConnection(newSocket, remoteId)); waitViewLock.unlock(); } else { + logger.debug("Trying establish connection with Replica: {}", remoteId); establishConnection(newSocket, remoteId); } //******* EDUARDO END **************// } catch (SocketTimeoutException ex) { - - logger.debug("Server socket timed out, retrying"); - } catch (IOException ex) { - logger.error("Problem during thread execution", ex); - } + logger.trace("Server socket timed out, retrying"); + } catch (SSLHandshakeException sslex) { + sslex.printStackTrace(); + } catch (IOException ex) { + logger.error("Problem during thread execution", ex); + } } try { @@ -288,51 +363,55 @@ public void run() { } //******* EDUARDO BEGIN **************// - private void establishConnection(Socket newSocket, int remoteId) throws IOException { + private void establishConnection(SSLSocket newSocket, int remoteId) throws IOException { if ((this.controller.getStaticConf().getTTPId() == remoteId) || this.controller.isCurrentViewMember(remoteId)) { connectionsLock.lock(); - //System.out.println("Vai se conectar com: "+remoteId); if (this.connections.get(remoteId) == null) { //This must never happen!!! //first time that this connection is being established //System.out.println("THIS DOES NOT HAPPEN....."+remoteId); - this.connections.put(remoteId, new ServerConnection(controller, newSocket, remoteId, inQueue, replica)); + this.connections.put(remoteId, + new ServerConnection(controller, newSocket, remoteId, inQueue, replica)); } else { - //reconnection + //reconnection + logger.debug("ReConnecting with replica: {}", remoteId); this.connections.get(remoteId).reconnect(newSocket); } connectionsLock.unlock(); } else { - //System.out.println("Closing connection of: "+remoteId); + logger.debug("Closing connection with replica: {}", remoteId); newSocket.close(); } } //******* EDUARDO END **************// + public static void setSSLSocketOptions(SSLSocket socket) { + try { + socket.setTcpNoDelay(true); + } catch (SocketException ex) { + LoggerFactory.getLogger(ServersCommunicationLayer.class). + error("Failed to set TCPNODELAY", ex); + } + } + public static void setSocketOptions(Socket socket) { try { socket.setTcpNoDelay(true); } catch (SocketException ex) { - - LoggerFactory.getLogger(ServersCommunicationLayer.class).error("Failed to set TCPNODELAY", ex); + LoggerFactory.getLogger(ServersCommunicationLayer.class) + .error("Failed to set TCPNODELAY", ex); } } @Override public String toString() { String str = "inQueue=" + inQueue.toString(); - int[] activeServers = controller.getCurrentViewAcceptors(); - for (int i = 0; i < activeServers.length; i++) { - - //for(int i=0; i()); - } - } - } - - /** - * Called when a WRITE message is received - * - * @param epoch Epoch of the receives message - * @param a Replica that sent the message - * @param value Value sent in the message - */ - private void writeReceived(Epoch epoch, int a, byte[] value) { - int cid = epoch.getConsensus().getId(); - logger.debug("WRITE from " + a + " for consensus " + cid); - epoch.setWrite(a, value); - - computeWrite(cid, epoch, value); - } - - /** - * Computes WRITE values according to Byzantine consensus specification - * values received). - * - * @param cid Consensus ID of the received message - * @param epoch Epoch of the receives message - * @param value Value sent in the message - */ - private void computeWrite(int cid, Epoch epoch, byte[] value) { - int writeAccepted = epoch.countWrite(value); - - logger.debug("I have " + writeAccepted + - " WRITEs for " + cid + "," + epoch.getTimestamp()); - - if (writeAccepted > controller.getQuorum() && Arrays.equals(value, epoch.propValueHash)) { - - if (!epoch.isAcceptSetted(me)) { - - logger.debug("Sending WRITE for " + cid); - - /**** LEADER CHANGE CODE! ******/ - logger.debug("Setting consensus " + cid + " QuorumWrite tiemstamp to " + epoch.getConsensus().getEts() + " and value " + Arrays.toString(value)); - epoch.getConsensus().setQuorumWrites(value); - /*****************************************/ - - epoch.setAccept(me, value); - - if(epoch.getConsensus().getDecision().firstMessageProposed!=null) { - - epoch.getConsensus().getDecision().firstMessageProposed.acceptSentTime = System.nanoTime(); - } - - ConsensusMessage cm = factory.createAccept(cid, epoch.getTimestamp(), value); - - // Create a cryptographic proof for this ACCEPT message - logger.debug("Creating cryptographic proof for my ACCEPT message from consensus " + cid); - insertProof(cm, epoch); - - int[] targets = this.controller.getCurrentViewOtherAcceptors(); - communication.getServersConn().send(targets, cm, true); - - //communication.send(this.reconfManager.getCurrentViewOtherAcceptors(), - //factory.createStrong(cid, epoch.getNumber(), value)); - epoch.addToProof(cm); - computeAccept(cid, epoch, value); - } - } - } - - /** - * Create a cryptographic proof for a consensus message - * - * This method modifies the consensus message passed as an argument, - * so that it contains a cryptographic proof. - * - * @param cm The consensus message to which the proof shall be set - * @param epoch The epoch during in which the consensus message was created - */ - private void insertProof(ConsensusMessage cm, Epoch epoch) { - ByteArrayOutputStream bOut = new ByteArrayOutputStream(248); - try { - new ObjectOutputStream(bOut).writeObject(cm); - } catch (IOException ex) { - logger.error("Failed to serialize consensus message",ex); - } - - byte[] data = bOut.toByteArray(); - - // check if consensus contains reconfiguration request - TOMMessage[] msgs = epoch.deserializedPropValue; - boolean hasReconf = false; - - for (TOMMessage msg : msgs) { - if (msg.getReqType() == TOMMessageType.RECONFIG - && msg.getViewID() == controller.getCurrentViewId()) { - hasReconf = true; - break; // no need to continue, exit the loop - } - } - - //If this consensus contains a reconfiguration request, we need to use - // signatures (there might be replicas that will not be part of the next - //consensus instance, and so their MAC will be outdated and useless) - if (hasReconf) { - - PrivateKey privKey = controller.getStaticConf().getPrivateKey(); - - byte[] signature = TOMUtil.signMessage(privKey, data); - - cm.setProof(signature); - - } else { //... if not, we can use MAC vectores - int[] processes = this.controller.getCurrentViewAcceptors(); - - HashMap macVector = new HashMap<>(); - - for (int id : processes) { - - try { - - SecretKey key = null; - do { - key = communication.getServersConn().getSecretKey(id); - if (key == null) { - logger.warn("I don't have yet a secret key with " + id + ". Retrying."); - Thread.sleep(1000); - } - - } while (key == null); // JCS: This loop is to solve a race condition where a - // replica might have already been inserted in the view or - // recovered after a crash, but it still did not concluded - // the diffie helman protocol. Not an elegant solution, - // but for now it will do - this.mac.init(key); - macVector.put(id, this.mac.doFinal(data)); - } catch (InterruptedException ex) { - - logger.error("Interruption while sleeping", ex); - } catch (InvalidKeyException ex) { - - logger.error("Failed to generate MAC vector", ex); - } - } - - cm.setProof(macVector); - } - - } - - /** - * Called when a ACCEPT message is received - * @param epoch Epoch of the receives message - * @param a Replica that sent the message - * @param value Value sent in the message - */ - private void acceptReceived(Epoch epoch, ConsensusMessage msg) { - int cid = epoch.getConsensus().getId(); - logger.debug("ACCEPT from " + msg.getSender() + " for consensus " + cid); - epoch.setAccept(msg.getSender(), msg.getValue()); - epoch.addToProof(msg); - - computeAccept(cid, epoch, msg.getValue()); - } - - /** - * Computes ACCEPT values according to the Byzantine consensus - * specification - * @param epoch Epoch of the receives message - * @param value Value sent in the message - */ - private void computeAccept(int cid, Epoch epoch, byte[] value) { - logger.debug("I have " + epoch.countAccept(value) + - " ACCEPTs for " + cid + "," + epoch.getTimestamp()); - - if (epoch.countAccept(value) > controller.getQuorum() && !epoch.getConsensus().isDecided()) { - logger.debug("Deciding consensus " + cid); - decide(epoch); - } - } - - /** - * This is the method invoked when a value is decided by this process - * @param epoch Epoch at which the decision is made - */ - private void decide(Epoch epoch) { - if (epoch.getConsensus().getDecision().firstMessageProposed != null) - epoch.getConsensus().getDecision().firstMessageProposed.decisionTime = System.nanoTime(); - - epoch.getConsensus().decided(epoch, true); - } + + private Logger logger = LoggerFactory.getLogger(this.getClass()); + + private int me; // This replica ID + private ExecutionManager executionManager; // Execution manager of consensus's executions + private MessageFactory factory; // Factory for PaW messages + private ServerCommunicationSystem communication; // Replicas comunication system + private TOMLayer tomLayer; // TOM layer + private ServerViewController controller; + + // thread pool used to paralelise creation of consensus proofs + private ExecutorService proofExecutor = null; + + /** + * Tulio Ribeiro + */ + private PrivateKey privKey; + + /** + * Creates a new instance of Acceptor. + * + * @param communication Replicas communication system + * @param factory Message factory for PaW messages + * @param controller + */ + public Acceptor(ServerCommunicationSystem communication, MessageFactory factory, ServerViewController controller) { + this.communication = communication; + this.me = controller.getStaticConf().getProcessId(); + this.factory = factory; + this.controller = controller; + + /* Tulio Ribeiro */ + this.privKey = controller.getStaticConf().getPrivateKey(); + + // use either the same number of Netty workers threads if specified in the + // configuration + // or use a many as the number of cores available + /* + * int nWorkers = this.controller.getStaticConf().getNumNettyWorkers(); nWorkers + * = nWorkers > 0 ? nWorkers : Runtime.getRuntime().availableProcessors(); + * this.proofExecutor = Executors.newWorkStealingPool(nWorkers); + */ + this.proofExecutor = Executors.newSingleThreadExecutor(); + } + + public MessageFactory getFactory() { + return factory; + } + + /** + * Sets the execution manager for this acceptor + * + * @param manager Execution manager for this acceptor + */ + public void setExecutionManager(ExecutionManager manager) { + this.executionManager = manager; + } + + /** + * Sets the TOM layer for this acceptor + * + * @param tom TOM layer for this acceptor + */ + public void setTOMLayer(TOMLayer tom) { + this.tomLayer = tom; + } + + /** + * Called by communication layer to delivery Paxos messages. This method only + * verifies if the message can be executed and calls process message (storing it + * on an out of context message buffer if this is not the case) + * + * @param msg Paxos messages delivered by the communication layer + */ + public final void deliver(ConsensusMessage msg) { + if (executionManager.checkLimits(msg)) { + logger.debug("Processing paxos msg with id " + msg.getNumber()); + processMessage(msg); + } else { + logger.debug("Out of context msg with id " + msg.getNumber()); + tomLayer.processOutOfContext(); + } + } + + /** + * Called when a Consensus message is received or when a out of context message + * must be processed. It processes the received message according to its type + * + * @param msg The message to be processed + */ + public final void processMessage(ConsensusMessage msg) { + Consensus consensus = executionManager.getConsensus(msg.getNumber()); + + consensus.lock.lock(); + Epoch epoch = consensus.getEpoch(msg.getEpoch(), controller); + switch (msg.getType()) { + case MessageFactory.PROPOSE: { + proposeReceived(epoch, msg); + } + break; + case MessageFactory.WRITE: { + writeReceived(epoch, msg.getSender(), msg.getValue()); + } + break; + case MessageFactory.ACCEPT: { + acceptReceived(epoch, msg); + } + } + consensus.lock.unlock(); + } + + /** + * Called when a PROPOSE message is received or when processing a formerly out + * of context propose which is know belongs to the current consensus. + * + * @param msg The PROPOSE message to by processed + */ + public void proposeReceived(Epoch epoch, ConsensusMessage msg) { + int cid = epoch.getConsensus().getId(); + int ts = epoch.getConsensus().getEts(); + int ets = executionManager.getConsensus(msg.getNumber()).getEts(); + logger.debug("PROPOSE received from:{}, for consensus cId:{}, I am:{}", msg.getSender(), cid, me); + if (msg.getSender() == executionManager.getCurrentLeader() // Is the replica the leader? + && epoch.getTimestamp() == 0 && ts == ets && ets == 0) { // Is all this in epoch 0? + executePropose(epoch, msg.getValue()); + } else { + logger.debug("Propose received is not from the expected leader"); + } + } + + /** + * Executes actions related to a proposed value. + * + * @param epoch the current epoch of the consensus + * @param value Value that is proposed + */ + private void executePropose(Epoch epoch, byte[] value) { + int cid = epoch.getConsensus().getId(); + logger.debug("Executing propose for cId:{}, Epoch Timestamp:{}", cid, epoch.getTimestamp()); + + long consensusStartTime = System.nanoTime(); + + if (epoch.propValue == null) { // only accept one propose per epoch + epoch.propValue = value; + epoch.propValueHash = tomLayer.computeHash(value); + + /*** LEADER CHANGE CODE ********/ + epoch.getConsensus().addWritten(value); + logger.trace("I have written value " + Arrays.toString(epoch.propValueHash) + " in consensus instance " + + cid + " with timestamp " + epoch.getConsensus().getEts()); + /*****************************************/ + + // start this consensus if it is not already running + if (cid == tomLayer.getLastExec() + 1) { + tomLayer.setInExec(cid); + } + epoch.deserializedPropValue = tomLayer.checkProposedValue(value, true); + + if (epoch.deserializedPropValue != null && !epoch.isWriteSent()) { + if (epoch.getConsensus().getDecision().firstMessageProposed == null) { + epoch.getConsensus().getDecision().firstMessageProposed = epoch.deserializedPropValue[0]; + } + if (epoch.getConsensus().getDecision().firstMessageProposed.consensusStartTime == 0) { + epoch.getConsensus().getDecision().firstMessageProposed.consensusStartTime = consensusStartTime; + + } + epoch.getConsensus().getDecision().firstMessageProposed.proposeReceivedTime = System.nanoTime(); + + if (controller.getStaticConf().isBFT()) { + logger.debug("Sending WRITE for " + cid); + + epoch.setWrite(me, epoch.propValueHash); + epoch.getConsensus().getDecision().firstMessageProposed.writeSentTime = System.nanoTime(); + + logger.debug("Sending WRITE for cId:{}, I am:{}", cid, me); + communication.send(this.controller.getCurrentViewOtherAcceptors(), + factory.createWrite(cid, epoch.getTimestamp(), epoch.propValueHash)); + + epoch.writeSent(); + + computeWrite(cid, epoch, epoch.propValueHash); + + logger.debug("WRITE computed for cId:{}, I am:{}", cid, me); + + } else { + epoch.setAccept(me, epoch.propValueHash); + epoch.getConsensus().getDecision().firstMessageProposed.writeSentTime = System.nanoTime(); + epoch.getConsensus().getDecision().firstMessageProposed.acceptSentTime = System.nanoTime(); + + /**** LEADER CHANGE CODE! ******/ + logger.debug("[CFT Mode] Setting consensus " + cid + " QuorumWrite tiemstamp to " + + epoch.getConsensus().getEts() + " and value " + Arrays.toString(epoch.propValueHash)); + epoch.getConsensus().setQuorumWrites(epoch.propValueHash); + /*****************************************/ + + communication.send(this.controller.getCurrentViewOtherAcceptors(), + factory.createAccept(cid, epoch.getTimestamp(), epoch.propValueHash)); + + epoch.acceptSent(); + computeAccept(cid, epoch, epoch.propValueHash); + } + executionManager.processOutOfContext(epoch.getConsensus()); + + } else if (epoch.deserializedPropValue == null + && !tomLayer.isChangingLeader()) { // force a leader change + tomLayer.getSynchronizer().triggerTimeout(new LinkedList<>()); + } + } + } + + /** + * Called when a WRITE message is received + * + * @param epoch Epoch of the receives message + * @param a Replica that sent the message + * @param value Value sent in the message + */ + private void writeReceived(Epoch epoch, int sender, byte[] value) { + int cid = epoch.getConsensus().getId(); + logger.debug("WRITE received from:{}, for consensus cId:{}", + sender, cid); + epoch.setWrite(sender, value); + + computeWrite(cid, epoch, value); + } + + /** + * Computes WRITE values according to Byzantine consensus specification values + * received). + * + * @param cid Consensus ID of the received message + * @param epoch Epoch of the receives message + * @param value Value sent in the message + */ + private void computeWrite(int cid, Epoch epoch, byte[] value) { + int writeAccepted = epoch.countWrite(value); + + logger.debug("I have {}, WRITE's for cId:{}, Epoch timestamp:{},", writeAccepted, cid, epoch.getTimestamp()); + + if (writeAccepted > controller.getQuorum() + && Arrays.equals(value, epoch.propValueHash)) { + + if (!epoch.isAcceptSent()) { + + logger.debug("Sending ACCEPT message, cId:{}, I am:{}", cid, me); + + /**** LEADER CHANGE CODE! ******/ + logger.debug("Setting consensus " + cid + " QuorumWrite tiemstamp to " + epoch.getConsensus().getEts() + + " and value " + Arrays.toString(value)); + epoch.getConsensus().setQuorumWrites(value); + /*****************************************/ + + if (epoch.getConsensus().getDecision().firstMessageProposed != null) { + + epoch.getConsensus().getDecision().firstMessageProposed.acceptSentTime = System.nanoTime(); + } + + ConsensusMessage cm = epoch.fetchAccept(); + int[] targets = this.controller.getCurrentViewAcceptors(); + epoch.acceptSent(); + + if (Arrays.equals(cm.getValue(), value)) { // make sure the ACCEPT message generated upon receiving the + // PROPOSE message + // still matches the value that ended up being written... + + logger.debug( + "Speculative ACCEPT message for consensus {} matches the written value, sending it to the other replicas", + cid); + + communication.getServersConn().send(targets, cm, true); + + } else { // ... and if not, create the ACCEPT message again (with the correct value), and + // send it + + ConsensusMessage correctAccept = factory.createAccept(cid, epoch.getTimestamp(), value); + + proofExecutor.submit(() -> { + + // Create a cryptographic proof for this ACCEPT message + logger.debug( + "Creating cryptographic proof for the correct ACCEPT message from consensus " + cid); + insertProof(correctAccept, epoch.deserializedPropValue); + + communication.getServersConn().send(targets, correctAccept, true); + + }); + } + + } + + } else if (!epoch.isAcceptCreated()) { // start creating the ACCEPT message and its respective proof ASAP, to + // increase performance. + // since this is done after a PROPOSE message is received, this is done + // speculatively, hence + // the value must be verified before sending the ACCEPT message to the + // other replicas + + ConsensusMessage cm = factory.createAccept(cid, epoch.getTimestamp(), value); + epoch.acceptCreated(); + + proofExecutor.submit(() -> { + + // Create a cryptographic proof for this ACCEPT message + logger.debug("Creating cryptographic proof for speculative ACCEPT message from consensus " + cid); + insertProof(cm, epoch.deserializedPropValue); + + epoch.setAcceptMsg(cm); + + }); + } + } + + /** + * Create a cryptographic proof for a consensus message + * + * This method modifies the consensus message passed as an argument, so that it + * contains a cryptographic proof. + * + * @param cm The consensus message to which the proof shall be set + * @param epoch The epoch during in which the consensus message was created + */ + private void insertProof(ConsensusMessage cm, TOMMessage[] msgs) { + ByteArrayOutputStream bOut = new ByteArrayOutputStream(248); + try { + ObjectOutputStream obj = new ObjectOutputStream(bOut); + obj.writeObject(cm); + obj.flush(); + bOut.flush(); + } catch (IOException ex) { + logger.error("Failed to serialize consensus message", ex); + } + + byte[] data = bOut.toByteArray(); + + // Always sign a consensus proof. + byte[] signature = TOMUtil.signMessage(privKey, data); + + cm.setProof(signature); + + } + + /** + * Called when a ACCEPT message is received + * + * @param epoch Epoch of the receives message + * @param a Replica that sent the message + * @param value Value sent in the message + */ + private void acceptReceived(Epoch epoch, ConsensusMessage msg) { + int cid = epoch.getConsensus().getId(); + logger.debug("ACCEPT from " + msg.getSender() + " for consensus " + cid); + epoch.setAccept(msg.getSender(), msg.getValue()); + epoch.addToProof(msg); + + computeAccept(cid, epoch, msg.getValue()); + } + + /** + * Computes ACCEPT values according to the Byzantine consensus specification + * + * @param epoch Epoch of the receives message + * @param value Value sent in the message + */ + private void computeAccept(int cid, Epoch epoch, byte[] value) { + logger.debug("I have {} ACCEPTs for cId:{}, Timestamp:{} ", epoch.countAccept(value), cid, + epoch.getTimestamp()); + + if (epoch.countAccept(value) > controller.getQuorum() && !epoch.getConsensus().isDecided()) { + logger.debug("Deciding consensus " + cid); + decide(epoch); + } + } + + /** + * This is the method invoked when a value is decided by this process + * + * @param epoch Epoch at which the decision is made + */ + private void decide(Epoch epoch) { + if (epoch.getConsensus().getDecision().firstMessageProposed != null) + epoch.getConsensus().getDecision().firstMessageProposed.decisionTime = System.nanoTime(); + + epoch.getConsensus().decided(epoch, true); + } } diff --git a/src/bftsmart/demo/map/MapClient.java b/src/bftsmart/demo/map/MapClient.java new file mode 100644 index 000000000..58f8e12f9 --- /dev/null +++ b/src/bftsmart/demo/map/MapClient.java @@ -0,0 +1,190 @@ +package bftsmart.demo.map; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.ObjectInput; +import java.io.ObjectInputStream; +import java.io.ObjectOutput; +import java.io.ObjectOutputStream; +import java.util.Collection; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; + +import bftsmart.tom.ServiceProxy; + +public class MapClient implements Map{ + + ServiceProxy serviceProxy; + + public MapClient(int clientId) { + serviceProxy = new ServiceProxy(clientId); + } + + @SuppressWarnings("unchecked") + @Override + public V put(K key, V value) { + try (ByteArrayOutputStream byteOut = new ByteArrayOutputStream(); + ObjectOutput objOut = new ObjectOutputStream(byteOut);) { + + objOut.writeObject(MapRequestType.PUT); + objOut.writeObject(key); + objOut.writeObject(value); + + objOut.flush(); + byteOut.flush(); + + byte[] reply = serviceProxy.invokeOrdered(byteOut.toByteArray()); + if (reply.length == 0) + return null; + try (ByteArrayInputStream byteIn = new ByteArrayInputStream(reply); + ObjectInput objIn = new ObjectInputStream(byteIn)) { + return (V)objIn.readObject(); + } + + } catch (IOException | ClassNotFoundException e) { + System.out.println("Exception putting value into map: " + e.getMessage()); + } + return null; + } + + @SuppressWarnings("unchecked") + @Override + public V get(Object key) { + try (ByteArrayOutputStream byteOut = new ByteArrayOutputStream(); + ObjectOutput objOut = new ObjectOutputStream(byteOut);) { + + objOut.writeObject(MapRequestType.GET); + objOut.writeObject(key); + + objOut.flush(); + byteOut.flush(); + + byte[] reply = serviceProxy.invokeUnordered(byteOut.toByteArray()); + if (reply.length == 0) + return null; + try (ByteArrayInputStream byteIn = new ByteArrayInputStream(reply); + ObjectInput objIn = new ObjectInputStream(byteIn)) { + return (V)objIn.readObject(); + } + + } catch (IOException | ClassNotFoundException e) { + System.out.println("Exception getting value from map: " + e.getMessage()); + } + return null; + } + + @SuppressWarnings("unchecked") + @Override + public V remove(Object key) { + try (ByteArrayOutputStream byteOut = new ByteArrayOutputStream(); + ObjectOutput objOut = new ObjectOutputStream(byteOut);) { + + objOut.writeObject(MapRequestType.REMOVE); + objOut.writeObject(key); + + objOut.flush(); + byteOut.flush(); + + byte[] reply = serviceProxy.invokeOrdered(byteOut.toByteArray()); + if (reply.length == 0) + return null; + try (ByteArrayInputStream byteIn = new ByteArrayInputStream(reply); + ObjectInput objIn = new ObjectInputStream(byteIn)) { + return (V)objIn.readObject(); + } + + } catch (IOException | ClassNotFoundException e) { + System.out.println("Exception removing value from map: " + e.getMessage()); + } + return null; + } + + @Override + public int size() { + try (ByteArrayOutputStream byteOut = new ByteArrayOutputStream(); + ObjectOutput objOut = new ObjectOutputStream(byteOut);) { + objOut.writeObject(MapRequestType.SIZE); + objOut.flush(); + byteOut.flush(); + + byte[] reply = serviceProxy.invokeUnordered(byteOut.toByteArray()); + try (ByteArrayInputStream byteIn = new ByteArrayInputStream(reply); + ObjectInput objIn = new ObjectInputStream(byteIn)) { + return objIn.readInt(); + } + + } catch (IOException e) { + System.out.println("Exception reading size of map: " + e.getMessage()); + } + return -1; + } + + @SuppressWarnings("unchecked") + @Override + public Set keySet() { + try (ByteArrayOutputStream byteOut = new ByteArrayOutputStream(); + ObjectOutput objOut = new ObjectOutputStream(byteOut);) { + + objOut.writeObject(MapRequestType.KEYSET); + + objOut.flush(); + byteOut.flush(); + + byte[] reply = serviceProxy.invokeUnordered(byteOut.toByteArray()); + try (ByteArrayInputStream byteIn = new ByteArrayInputStream(reply); + ObjectInput objIn = new ObjectInputStream(byteIn)) { + int size = objIn.readInt(); + Set result = new HashSet<>(); + while (size-- > 0) { + result.add((K)objIn.readObject()); + } + return result; + } + + } catch (IOException | ClassNotFoundException e) { + System.out.println("Exception getting keyset from map: " + e.getMessage()); + } + return null; + } + + public void close() { + serviceProxy.close(); + } + + @Override + public void clear() { + throw new UnsupportedOperationException("Not supported yet."); + } + + @Override + public boolean containsKey(Object key) { + throw new UnsupportedOperationException("Not supported yet."); + } + + @Override + public boolean containsValue(Object value) { + throw new UnsupportedOperationException("Not supported yet."); + } + + @Override + public Set> entrySet() { + throw new UnsupportedOperationException("Not supported yet."); + } + + @Override + public boolean isEmpty() { + throw new UnsupportedOperationException("Not supported yet."); + } + + @Override + public void putAll(Map m) { + throw new UnsupportedOperationException("Not supported yet."); + } + + @Override + public Collection values() { + throw new UnsupportedOperationException("Not supported yet."); + } +} \ No newline at end of file diff --git a/src/bftsmart/demo/map/MapInteractiveClient.java b/src/bftsmart/demo/map/MapInteractiveClient.java new file mode 100644 index 000000000..e6abef0eb --- /dev/null +++ b/src/bftsmart/demo/map/MapInteractiveClient.java @@ -0,0 +1,72 @@ +package bftsmart.demo.map; + +import java.io.Console; +import java.util.Set; + +public class MapInteractiveClient { + + public static void main(String[] args) { + if(args.length < 1) { + System.out.println("Usage: demo.map.MapInteractiveClient "); + } + + int clientId = Integer.parseInt(args[0]); + MapClient map = new MapClient<>(clientId); + Console console = System.console(); + + boolean exit = false; + String key, value, result; + while(!exit) { + System.out.println("Select an option:"); + System.out.println("0 - Terminate this client"); + System.out.println("1 - Insert value into the map"); + System.out.println("2 - Retrieve value from the map"); + System.out.println("3 - Removes value from the map"); + System.out.println("4 - Retrieve the size of the map"); + System.out.println("5 - List all keys available in the table"); + + int cmd = Integer.parseInt(console.readLine("Option:")); + + switch (cmd) { + case 0: + map.close(); + exit = true; + break; + case 1: + System.out.println("Putting value in the map"); + key = console.readLine("Enter the key:"); + value = console.readLine("Enter the value:"); + result = map.put(key, value); + System.out.println("Previous value: " + result); + break; + case 2: + System.out.println("Reading value from the map"); + key = console.readLine("Enter the key:"); + result = map.get(key); + System.out.println("Value read: " + result); + break; + case 3: + System.out.println("Removing value in the map"); + key = console.readLine("Enter the key:"); + result = map.remove(key); + System.out.println("Value removed: " + result); + break; + case 4: + System.out.println("Getting the map size"); + int size = map.size(); + System.out.println("Map size: " + size); + break; + case 5: + System.out.println("Getting all keys"); + Set keys = map.keySet(); + System.out.println("Total number of keys found: " + keys.size()); + for (String k : keys) + System.out.println("---> " + k); + break; + default: + break; + } + } + } + +} diff --git a/src/bftsmart/demo/map/MapRequestType.java b/src/bftsmart/demo/map/MapRequestType.java new file mode 100644 index 000000000..a5f910aa8 --- /dev/null +++ b/src/bftsmart/demo/map/MapRequestType.java @@ -0,0 +1,10 @@ +package bftsmart.demo.map; + +/** + * Types of operation for BFT Map implementation + * + */ + +public enum MapRequestType { + PUT, GET, SIZE, REMOVE, KEYSET; +} \ No newline at end of file diff --git a/src/bftsmart/demo/map/MapServer.java b/src/bftsmart/demo/map/MapServer.java new file mode 100644 index 000000000..1ea59c04c --- /dev/null +++ b/src/bftsmart/demo/map/MapServer.java @@ -0,0 +1,180 @@ +package bftsmart.demo.map; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.ObjectInput; +import java.io.ObjectInputStream; +import java.io.ObjectOutput; +import java.io.ObjectOutputStream; +import java.util.Map; +import java.util.Set; +import java.util.TreeMap; +import java.util.logging.Level; +import java.util.logging.Logger; + +import bftsmart.tom.MessageContext; +import bftsmart.tom.ServiceReplica; +import bftsmart.tom.server.defaultservices.DefaultSingleRecoverable; + +public class MapServer extends DefaultSingleRecoverable { + + private Map replicaMap; + private Logger logger; + + public MapServer(int id) { + replicaMap = new TreeMap<>(); + logger = Logger.getLogger(MapServer.class.getName()); + new ServiceReplica(id, this, this); + } + + public static void main(String[] args) { + if (args.length < 1) { + System.out.println("Usage: demo.map.MapServer "); + System.exit(-1); + } + new MapServer(Integer.parseInt(args[0])); + } + + @SuppressWarnings("unchecked") + @Override + public byte[] appExecuteOrdered(byte[] command, MessageContext msgCtx) { + byte[] reply = null; + K key = null; + V value = null; + boolean hasReply = false; + try (ByteArrayInputStream byteIn = new ByteArrayInputStream(command); + ObjectInput objIn = new ObjectInputStream(byteIn); + ByteArrayOutputStream byteOut = new ByteArrayOutputStream(); + ObjectOutput objOut = new ObjectOutputStream(byteOut);) { + MapRequestType reqType = (MapRequestType)objIn.readObject(); + switch (reqType) { + case PUT: + key = (K)objIn.readObject(); + value = (V)objIn.readObject(); + + V oldValue = replicaMap.put(key, value); + if (oldValue != null) { + objOut.writeObject(oldValue); + hasReply = true; + } + break; + case GET: + key = (K)objIn.readObject(); + value = replicaMap.get(key); + if (value != null) { + objOut.writeObject(value); + hasReply = true; + } + break; + case REMOVE: + key = (K)objIn.readObject(); + value = replicaMap.remove(key); + if (value != null) { + objOut.writeObject(value); + hasReply = true; + } + break; + case SIZE: + int size = replicaMap.size(); + objOut.writeInt(size); + hasReply = true; + break; + case KEYSET: + keySet(objOut); + hasReply = true; + break; + } + if (hasReply) { + objOut.flush(); + byteOut.flush(); + reply = byteOut.toByteArray(); + } else { + reply = new byte[0]; + } + + } catch (IOException | ClassNotFoundException e) { + logger.log(Level.SEVERE, "Ocurred during map operation execution", e); + } + return reply; + } + + @SuppressWarnings("unchecked") + @Override + public byte[] appExecuteUnordered(byte[] command, MessageContext msgCtx) { + byte[] reply = null; + K key = null; + V value = null; + boolean hasReply = false; + + try (ByteArrayInputStream byteIn = new ByteArrayInputStream(command); + ObjectInput objIn = new ObjectInputStream(byteIn); + ByteArrayOutputStream byteOut = new ByteArrayOutputStream(); + ObjectOutput objOut = new ObjectOutputStream(byteOut);) { + MapRequestType reqType = (MapRequestType)objIn.readObject(); + switch (reqType) { + case GET: + key = (K)objIn.readObject(); + value = replicaMap.get(key); + if (value != null) { + objOut.writeObject(value); + hasReply = true; + } + break; + case SIZE: + int size = replicaMap.size(); + objOut.writeInt(size); + hasReply = true; + break; + case KEYSET: + keySet(objOut); + hasReply = true; + break; + default: + logger.log(Level.WARNING, "in appExecuteUnordered only read operations are supported"); + } + if (hasReply) { + objOut.flush(); + byteOut.flush(); + reply = byteOut.toByteArray(); + } else { + reply = new byte[0]; + } + } catch (IOException | ClassNotFoundException e) { + logger.log(Level.SEVERE, "Ocurred during map operation execution", e); + } + + return reply; + } + + private void keySet(ObjectOutput out) throws IOException, ClassNotFoundException { + Set keySet = replicaMap.keySet(); + int size = replicaMap.size(); + out.writeInt(size); + for (K key : keySet) + out.writeObject(key); + } + + @Override + public byte[] getSnapshot() { + try (ByteArrayOutputStream byteOut = new ByteArrayOutputStream(); + ObjectOutput objOut = new ObjectOutputStream(byteOut)) { + objOut.writeObject(replicaMap); + return byteOut.toByteArray(); + } catch (IOException e) { + logger.log(Level.SEVERE, "Error while taking snapshot", e); + } + return new byte[0]; + } + + @SuppressWarnings("unchecked") + @Override + public void installSnapshot(byte[] state) { + try (ByteArrayInputStream byteIn = new ByteArrayInputStream(state); + ObjectInput objIn = new ObjectInputStream(byteIn)) { + replicaMap = (Map)objIn.readObject(); + } catch (IOException | ClassNotFoundException e) { + logger.log(Level.SEVERE, "Error while installing snapshot", e); + } + } +} \ No newline at end of file diff --git a/src/bftsmart/demo/microbenchmarks/AsyncLatencyClient.java b/src/bftsmart/demo/microbenchmarks/AsyncLatencyClient.java index 20bc05fa2..49063213d 100644 --- a/src/bftsmart/demo/microbenchmarks/AsyncLatencyClient.java +++ b/src/bftsmart/demo/microbenchmarks/AsyncLatencyClient.java @@ -24,8 +24,21 @@ import bftsmart.tom.core.messages.TOMMessage; import bftsmart.tom.core.messages.TOMMessageType; import bftsmart.tom.util.Storage; +import bftsmart.tom.util.TOMUtil; +import java.nio.ByteBuffer; +import java.security.InvalidKeyException; +import java.security.KeyFactory; +import java.security.NoSuchAlgorithmException; +import java.security.NoSuchProviderException; +import java.security.Security; +import java.security.Signature; +import java.security.SignatureException; +import java.security.spec.InvalidKeySpecException; +import java.security.spec.PKCS8EncodedKeySpec; +import java.util.Base64; import java.util.Collection; import java.util.LinkedList; +import java.util.Random; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Future; @@ -40,8 +53,8 @@ public class AsyncLatencyClient { static int initId; public static void main(String[] args) throws IOException { - if (args.length < 7) { - System.out.println("Usage: java ...AsyncLatencyClient "); + if (args.length < 8) { + System.out.println("Usage: ... ThroughputLatencyClient "); System.exit(-1); } @@ -52,6 +65,17 @@ public static void main(String[] args) throws IOException { int interval = Integer.parseInt(args[4]); boolean readOnly = Boolean.parseBoolean(args[5]); boolean verbose = Boolean.parseBoolean(args[6]); + String sign = args[7]; + + int s = 0; + if (!sign.equalsIgnoreCase("nosig")) s++; + if (sign.equalsIgnoreCase("ecdsa")) s++; + + if (s == 2 && Security.getProvider("SunEC") == null) { + + System.out.println("Option 'ecdsa' requires SunEC provider to be available."); + System.exit(0); + } Client[] clients = new Client[numThreads]; @@ -63,7 +87,7 @@ public static void main(String[] args) throws IOException { } System.out.println("Launching client " + (initId + i)); - clients[i] = new AsyncLatencyClient.Client(initId + i, numberOfOps, requestSize, interval, readOnly, verbose); + clients[i] = new AsyncLatencyClient.Client(initId + i, numberOfOps, requestSize, interval, readOnly, verbose, s); } ExecutorService exec = Executors.newFixedThreadPool(clients.length); @@ -94,21 +118,66 @@ static class Client extends Thread { int id; AsynchServiceProxy serviceProxy; int numberOfOps; + int requestSize; int interval; byte[] request; TOMMessageType reqType; boolean verbose; + Random rand; + int rampup = 3000; - public Client(int id, int numberOfOps, int requestSize, int interval, boolean readOnly, boolean verbose) { + public Client(int id, int numberOfOps, int requestSize, int interval, boolean readOnly, boolean verbose, int sign) { this.id = id; this.serviceProxy = new AsynchServiceProxy(id); this.numberOfOps = numberOfOps; + this.requestSize = requestSize; this.interval = interval; this.request = new byte[requestSize]; this.reqType = (readOnly ? TOMMessageType.UNORDERED_REQUEST : TOMMessageType.ORDERED_REQUEST); this.verbose = verbose; + this.request = new byte[this.requestSize]; + + rand = new Random(System.nanoTime() + this.id); + rand.nextBytes(request); + + byte[] signature = new byte[0]; + Signature eng; + + try { + + if (sign > 0) { + + if (sign == 1) { + eng = TOMUtil.getSigEngine(); + eng.initSign(serviceProxy.getViewManager().getStaticConf().getPrivateKey()); + } else { + + eng = Signature.getInstance("SHA256withECDSA", "SunEC"); + + KeyFactory kf = KeyFactory.getInstance("EC", "SunEC"); + Base64.Decoder b64 = Base64.getDecoder(); + PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(b64.decode(ThroughputLatencyClient.privKey)); + eng.initSign(kf.generatePrivate(spec)); + + } + eng.update(request); + signature = eng.sign(); + } + + ByteBuffer buffer = ByteBuffer.allocate(request.length + signature.length + (Integer.BYTES * 2)); + buffer.putInt(request.length); + buffer.put(request); + buffer.putInt(signature.length); + buffer.put(signature); + this.request = buffer.array(); + + + } catch (NoSuchAlgorithmException | SignatureException | NoSuchProviderException | InvalidKeyException | InvalidKeySpecException ex) { + ex.printStackTrace(); + System.exit(0); + } } @@ -153,8 +222,9 @@ public void replyReceived(RequestContext context, TOMMessage reply) { }, this.reqType); if (i > (this.numberOfOps / 2)) st.store(System.nanoTime() - last_send_instant); - if (this.interval > 0) { - Thread.sleep(this.interval); + if (this.interval > 0 || this.rampup > 0) { + Thread.sleep(Math.max(rand.nextInt(this.interval) + 1, this.rampup)); + if (this.rampup > 0) this.rampup -= 100; } if (this.verbose) System.out.println("Sending " + (i + 1) + "th op"); diff --git a/src/bftsmart/demo/microbenchmarks/LatencyClient.java b/src/bftsmart/demo/microbenchmarks/LatencyClient.java deleted file mode 100644 index 5b154b681..000000000 --- a/src/bftsmart/demo/microbenchmarks/LatencyClient.java +++ /dev/null @@ -1,88 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.demo.microbenchmarks; - -import java.io.IOException; - -import bftsmart.tom.ServiceProxy; -import bftsmart.tom.util.Storage; - -/** - * Example client that updates a BFT replicated service (a counter). - * - */ -public class LatencyClient { - - @SuppressWarnings("static-access") - public static void main(String[] args) throws IOException { - if (args.length < 5) { - System.out.println("Usage: java ...LatencyClient "); - System.exit(-1); - } - - ServiceProxy counterProxy = new ServiceProxy(Integer.parseInt(args[0])); - //counterProxy.setInvokeTimeout(1); - - try { - - int numberOfOps = Integer.parseInt(args[1]); - int requestSize = Integer.parseInt(args[2]); - int interval = Integer.parseInt(args[3]); - boolean readOnly = Boolean.parseBoolean(args[4]); - - byte[] request = new byte[requestSize], reply; - - System.out.println("Warm up..."); - - for (int i = 0; i < numberOfOps/2; i++) { - if(readOnly) - reply = counterProxy.invokeUnordered(request); - else - reply = counterProxy.invokeOrdered(request); - } - - Storage st = new Storage(numberOfOps/2); - - System.out.println("Executing experiment for "+numberOfOps/2+" ops"); - - for (int i = 0; i < numberOfOps/2; i++) { - long last_send_instant = System.nanoTime(); - if(readOnly) - reply = counterProxy.invokeUnordered(request); - else - reply = counterProxy.invokeOrdered(request); - st.store(System.nanoTime() - last_send_instant); - - if (interval > 0) { - //sleeps interval ms before sending next request - Thread.sleep(interval); - } - } - - System.out.println("Average time for " + numberOfOps / 2 + " executions (-10%) = " + st.getAverage(true) / 1000 + " us "); - System.out.println("Standard desviation for " + numberOfOps / 2 + " executions (-10%) = " + st.getDP(true) / 1000 + " us "); - System.out.println("Average time for " + numberOfOps / 2 + " executions (all samples) = " + st.getAverage(false) / 1000 + " us "); - System.out.println("Standard desviation for " + numberOfOps / 2 + " executions (all samples) = " + st.getDP(false) / 1000 + " us "); - System.out.println("Maximum time for " + numberOfOps / 2 + " executions (all samples) = " + st.getMax(false) / 1000 + " us "); - - } catch(Exception e){ - } finally { - counterProxy.close(); - } - - System.exit(0); - } -} diff --git a/src/bftsmart/demo/microbenchmarks/LatencyServer.java b/src/bftsmart/demo/microbenchmarks/LatencyServer.java deleted file mode 100644 index 07718af22..000000000 --- a/src/bftsmart/demo/microbenchmarks/LatencyServer.java +++ /dev/null @@ -1,172 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.demo.microbenchmarks; - -import bftsmart.statemanagement.StateManager; -import bftsmart.tom.MessageContext; -import bftsmart.tom.ReplicaContext; -import bftsmart.tom.ServiceReplica; -import bftsmart.tom.server.defaultservices.DefaultRecoverable; -import bftsmart.tom.util.Storage; - -/** - * Simple server that just acknowledge the reception of a request. - */ -public class LatencyServer extends DefaultRecoverable{ - - private int interval; - private int replySize; - - private int iterations = 0; - private Storage totalLatency = null; - private Storage consensusLatency = null; - private Storage preConsLatency = null; - private Storage posConsLatency = null; - private Storage proposeLatency = null; - private Storage writeLatency = null; - private Storage acceptLatency = null; - private ServiceReplica replica; - - public LatencyServer(int id, int interval, int replySize) { - - this.interval = interval; - this.replySize = replySize; - - totalLatency = new Storage(interval); - consensusLatency = new Storage(interval); - preConsLatency = new Storage(interval); - posConsLatency = new Storage(interval); - proposeLatency = new Storage(interval); - writeLatency = new Storage(interval); - acceptLatency = new Storage(interval); - - replica = new ServiceReplica(id, this, this); - } - - @Override - public byte[] appExecuteUnordered(byte[] command, MessageContext msgCtx) { - return execute(command,msgCtx); - } - - @Override - public byte[][] appExecuteBatch(byte[][] commands, MessageContext[] msgCtxs, boolean fromConsensus) { - byte[][] replies = new byte[commands.length][]; - - for (int i = 0; i < commands.length; i++) { - - replies[i] = execute(commands[i],msgCtxs[i]); - - } - - return replies; - } - - public byte[] execute(byte[] command, MessageContext msgCtx) { - boolean readOnly = false; - - iterations++; - - if (msgCtx != null && msgCtx.getFirstInBatch() != null) { - - - readOnly = msgCtx.readOnly; - - msgCtx.getFirstInBatch().executedTime = System.nanoTime(); - - totalLatency.store(msgCtx.getFirstInBatch().executedTime - msgCtx.getFirstInBatch().receptionTime); - - if (readOnly == false) { - - consensusLatency.store(msgCtx.getFirstInBatch().decisionTime - msgCtx.getFirstInBatch().consensusStartTime); - long temp = msgCtx.getFirstInBatch().consensusStartTime - msgCtx.getFirstInBatch().receptionTime; - preConsLatency.store(temp > 0 ? temp : 0); - posConsLatency.store(msgCtx.getFirstInBatch().executedTime - msgCtx.getFirstInBatch().decisionTime); - proposeLatency.store(msgCtx.getFirstInBatch().writeSentTime - msgCtx.getFirstInBatch().consensusStartTime); - writeLatency.store(msgCtx.getFirstInBatch().acceptSentTime - msgCtx.getFirstInBatch().writeSentTime); - acceptLatency.store(msgCtx.getFirstInBatch().decisionTime - msgCtx.getFirstInBatch().acceptSentTime); - - - } else { - - - consensusLatency.store(0); - preConsLatency.store(0); - posConsLatency.store(0); - proposeLatency.store(0); - writeLatency.store(0); - acceptLatency.store(0); - - - } - - } else { - - - consensusLatency.store(0); - preConsLatency.store(0); - posConsLatency.store(0); - proposeLatency.store(0); - writeLatency.store(0); - acceptLatency.store(0); - - - } - - if(iterations % interval == 0) { - System.out.println("--- Measurements after "+ iterations+" ops ("+interval+" samples) ---"); - System.out.println("Total latency = " + totalLatency.getAverage(false) / 1000 + " (+/- "+ (long)totalLatency.getDP(false) / 1000 +") us "); - totalLatency.reset(); - System.out.println("Consensus latency = " + consensusLatency.getAverage(false) / 1000 + " (+/- "+ (long)consensusLatency.getDP(false) / 1000 +") us "); - consensusLatency.reset(); - System.out.println("Pre-consensus latency = " + preConsLatency.getAverage(false) / 1000 + " (+/- "+ (long)preConsLatency.getDP(false) / 1000 +") us "); - preConsLatency.reset(); - System.out.println("Pos-consensus latency = " + posConsLatency.getAverage(false) / 1000 + " (+/- "+ (long)posConsLatency.getDP(false) / 1000 +") us "); - posConsLatency.reset(); - System.out.println("Propose latency = " + proposeLatency.getAverage(false) / 1000 + " (+/- "+ (long)proposeLatency.getDP(false) / 1000 +") us "); - proposeLatency.reset(); - System.out.println("Write latency = " + writeLatency.getAverage(false) / 1000 + " (+/- "+ (long)writeLatency.getDP(false) / 1000 +") us "); - writeLatency.reset(); - System.out.println("Accept latency = " + acceptLatency.getAverage(false) / 1000 + " (+/- "+ (long)acceptLatency.getDP(false) / 1000 +") us "); - acceptLatency.reset(); - } - - return new byte[replySize]; - } - - public static void main(String[] args){ - if(args.length < 3) { - System.out.println("Use: java ...LatencyServer "); - System.exit(-1); - } - - int processId = Integer.parseInt(args[0]); - int interval = Integer.parseInt(args[1]); - int replySize = Integer.parseInt(args[2]); - - new LatencyServer(processId,interval,replySize); - } - - @Override - public void installSnapshot(byte[] state) { - - } - - @Override - public byte[] getSnapshot() { - return new byte[0]; - } - -} diff --git a/src/bftsmart/demo/microbenchmarks/ThroughputLatencyClient.java b/src/bftsmart/demo/microbenchmarks/ThroughputLatencyClient.java index 48ebf5dde..91fbdb8f4 100644 --- a/src/bftsmart/demo/microbenchmarks/ThroughputLatencyClient.java +++ b/src/bftsmart/demo/microbenchmarks/ThroughputLatencyClient.java @@ -19,8 +19,21 @@ import bftsmart.tom.ServiceProxy; import bftsmart.tom.util.Storage; +import bftsmart.tom.util.TOMUtil; +import java.nio.ByteBuffer; +import java.security.InvalidKeyException; +import java.security.KeyFactory; +import java.security.NoSuchAlgorithmException; +import java.security.NoSuchProviderException; +import java.security.Security; +import java.security.Signature; +import java.security.SignatureException; +import java.security.spec.InvalidKeySpecException; +import java.security.spec.PKCS8EncodedKeySpec; +import java.util.Base64; import java.util.Collection; import java.util.LinkedList; +import java.util.Random; import java.util.concurrent.ExecutionException; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; @@ -34,10 +47,27 @@ public class ThroughputLatencyClient { public static int initId = 0; + public static String privKey = "MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgXa3mln4anewXtqrM" + + "hMw6mfZhslkRa/j9P790ToKjlsihRANCAARnxLhXvU4EmnIwhVl3Bh0VcByQi2um" + + "9KsJ/QdCDjRZb1dKg447voj5SZ8SSZOUglc/v8DJFFJFTfygjwi+27gz"; + + public static String pubKey = "MIICNjCCAd2gAwIBAgIRAMnf9/dmV9RvCCVw9pZQUfUwCgYIKoZIzj0EAwIwgYEx" + + "CzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1TYW4g" + + "RnJhbmNpc2NvMRkwFwYDVQQKExBvcmcxLmV4YW1wbGUuY29tMQwwCgYDVQQLEwND" + + "T1AxHDAaBgNVBAMTE2NhLm9yZzEuZXhhbXBsZS5jb20wHhcNMTcxMTEyMTM0MTEx" + + "WhcNMjcxMTEwMTM0MTExWjBpMQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZv" + + "cm5pYTEWMBQGA1UEBxMNU2FuIEZyYW5jaXNjbzEMMAoGA1UECxMDQ09QMR8wHQYD" + + "VQQDExZwZWVyMC5vcmcxLmV4YW1wbGUuY29tMFkwEwYHKoZIzj0CAQYIKoZIzj0D" + + "AQcDQgAEZ8S4V71OBJpyMIVZdwYdFXAckItrpvSrCf0HQg40WW9XSoOOO76I+Umf" + + "EkmTlIJXP7/AyRRSRU38oI8Ivtu4M6NNMEswDgYDVR0PAQH/BAQDAgeAMAwGA1Ud" + + "EwEB/wQCMAAwKwYDVR0jBCQwIoAginORIhnPEFZUhXm6eWBkm7K7Zc8R4/z7LW4H" + + "ossDlCswCgYIKoZIzj0EAwIDRwAwRAIgVikIUZzgfuFsGLQHWJUVJCU7pDaETkaz" + + "PzFgsCiLxUACICgzJYlW7nvZxP7b6tbeu3t8mrhMXQs956mD4+BoKuNI"; + @SuppressWarnings("static-access") public static void main(String[] args) throws IOException { - if (args.length < 7) { - System.out.println("Usage: ... ThroughputLatencyClient "); + if (args.length < 8) { + System.out.println("Usage: ... ThroughputLatencyClient "); System.exit(-1); } @@ -49,19 +79,30 @@ public static void main(String[] args) throws IOException { int interval = Integer.parseInt(args[4]); boolean readOnly = Boolean.parseBoolean(args[5]); boolean verbose = Boolean.parseBoolean(args[6]); + String sign = args[7]; + + int s = 0; + if (!sign.equalsIgnoreCase("nosig")) s++; + if (sign.equalsIgnoreCase("ecdsa")) s++; + + if (s == 2 && Security.getProvider("SunEC") == null) { + + System.out.println("Option 'ecdsa' requires SunEC provider to be available."); + System.exit(0); + } Client[] clients = new Client[numThreads]; for(int i=0; i 0) { + + if (sign == 1) { + eng = TOMUtil.getSigEngine(); + eng.initSign(proxy.getViewManager().getStaticConf().getPrivateKey()); + } else { + + eng = Signature.getInstance("SHA256withECDSA", "SunEC"); + + KeyFactory kf = KeyFactory.getInstance("EC", "SunEC"); + Base64.Decoder b64 = Base64.getDecoder(); + PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(b64.decode(ThroughputLatencyClient.privKey)); + eng.initSign(kf.generatePrivate(spec)); + + } + eng.update(request); + signature = eng.sign(); + } + + ByteBuffer buffer = ByteBuffer.allocate(request.length + signature.length + (Integer.BYTES * 2)); + buffer.putInt(request.length); + buffer.put(request); + buffer.putInt(signature.length); + buffer.put(signature); + this.request = buffer.array(); + + + } catch (NoSuchAlgorithmException | SignatureException | NoSuchProviderException | InvalidKeyException | InvalidKeySpecException ex) { + ex.printStackTrace(); + System.exit(0); + } + } public void run() { - + System.out.println("Warm up..."); int req = 0; diff --git a/src/bftsmart/demo/microbenchmarks/ThroughputLatencyServer.java b/src/bftsmart/demo/microbenchmarks/ThroughputLatencyServer.java index 1c73fe3bd..53864edb3 100644 --- a/src/bftsmart/demo/microbenchmarks/ThroughputLatencyServer.java +++ b/src/bftsmart/demo/microbenchmarks/ThroughputLatencyServer.java @@ -17,8 +17,34 @@ import bftsmart.tom.MessageContext; import bftsmart.tom.ServiceReplica; +import bftsmart.tom.server.defaultservices.CommandsInfo; import bftsmart.tom.server.defaultservices.DefaultRecoverable; import bftsmart.tom.util.Storage; +import bftsmart.tom.util.TOMUtil; +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.io.ObjectOutputStream; +import java.io.RandomAccessFile; +import java.nio.ByteBuffer; +import java.nio.channels.FileChannel; +import java.security.InvalidKeyException; +import java.security.KeyFactory; +import java.security.NoSuchAlgorithmException; +import java.security.NoSuchProviderException; +import java.security.Security; +import java.security.Signature; +import java.security.SignatureException; +import java.security.cert.CertificateException; +import java.security.cert.CertificateFactory; +import java.security.spec.InvalidKeySpecException; +import java.security.spec.PKCS8EncodedKeySpec; +import java.security.spec.X509EncodedKeySpec; +import java.util.Base64; +import java.util.logging.Level; +import java.util.logging.Logger; /** * Simple server that just acknowledge the reception of a request. @@ -26,9 +52,10 @@ public final class ThroughputLatencyServer extends DefaultRecoverable{ private int interval; - private int replySize; + private byte[] reply; private float maxTp = -1; private boolean context; + private int signed; private byte[] state; @@ -42,13 +69,25 @@ public final class ThroughputLatencyServer extends DefaultRecoverable{ private Storage proposeLatency = null; private Storage writeLatency = null; private Storage acceptLatency = null; + + private Storage batchSize = null; + private ServiceReplica replica; + + private RandomAccessFile randomAccessFile = null; + private FileChannel channel = null; - public ThroughputLatencyServer(int id, int interval, int replySize, int stateSize, boolean context) { + public ThroughputLatencyServer(int id, int interval, int replySize, int stateSize, boolean context, int signed, int write) { this.interval = interval; - this.replySize = replySize; + this.context = context; + this.signed = signed; + + this.reply = new byte[replySize]; + + for (int i = 0; i < replySize ;i++) + reply[i] = (byte) i; this.state = new byte[stateSize]; @@ -62,13 +101,37 @@ public ThroughputLatencyServer(int id, int interval, int replySize, int stateSiz proposeLatency = new Storage(interval); writeLatency = new Storage(interval); acceptLatency = new Storage(interval); - + + batchSize = new Storage(interval); + + if (write > 0) { + + try { + final File f = File.createTempFile("bft-"+id+"-", Long.toString(System.nanoTime())); + randomAccessFile = new RandomAccessFile(f, (write > 1 ? "rwd" : "rw")); + channel = randomAccessFile.getChannel(); + + Runtime.getRuntime().addShutdownHook(new Thread() { + + @Override + public void run() { + + f.delete(); + } + }); + } catch (IOException ex) { + ex.printStackTrace(); + System.exit(0); + } + } replica = new ServiceReplica(id, this, this); } @Override public byte[][] appExecuteBatch(byte[][] commands, MessageContext[] msgCtxs, boolean fromConsensus) { + batchSize.store(commands.length); + byte[][] replies = new byte[commands.length][]; for (int i = 0; i < commands.length; i++) { @@ -77,6 +140,37 @@ public byte[][] appExecuteBatch(byte[][] commands, MessageContext[] msgCtxs, boo } + if (randomAccessFile != null) { + + ObjectOutputStream oos = null; + try { + CommandsInfo cmd = new CommandsInfo(commands,msgCtxs); + ByteArrayOutputStream bos = new ByteArrayOutputStream(); + oos = new ObjectOutputStream(bos); + oos.writeObject(cmd); + oos.flush(); + byte[] bytes = bos.toByteArray(); + oos.close(); + bos.close(); + + ByteBuffer bb = ByteBuffer.allocate(bytes.length); + bb.put(bytes); + bb.flip(); + + channel.write(bb); + channel.force(false); + } catch (IOException ex) { + Logger.getLogger(ThroughputLatencyServer.class.getName()).log(Level.SEVERE, null, ex); + + } finally { + try { + oos.close(); + } catch (IOException ex) { + Logger.getLogger(ThroughputLatencyServer.class.getName()).log(Level.SEVERE, null, ex); + } + } + } + return replies; } @@ -163,15 +257,152 @@ public byte[] execute(byte[] command, MessageContext msgCtx) { System.out.println("Accept latency = " + acceptLatency.getAverage(false) / 1000 + " (+/- "+ (long)acceptLatency.getDP(false) / 1000 +") us "); acceptLatency.reset(); + System.out.println("Batch average size = " + batchSize.getAverage(false) + " (+/- "+ (long)batchSize.getDP(false) +") requests"); + batchSize.reset(); + throughputMeasurementStartTime = System.currentTimeMillis(); } - return new byte[replySize]; + return reply; } + + /*public byte[] execute(byte[] command, MessageContext msgCtx) { + + ByteBuffer buffer = ByteBuffer.wrap(command); + int l = buffer.getInt(); + byte[] request = new byte[l]; + buffer.get(request); + l = buffer.getInt(); + byte[] signature = new byte[l]; + + buffer.get(signature); + Signature eng; + + try { + + if (signed > 0) { + + if (signed == 1) { + + eng = TOMUtil.getSigEngine(); + eng.initVerify(replica.getReplicaContext().getStaticConfiguration().getPublicKey()); + } else { + + eng = Signature.getInstance("SHA256withECDSA", "SunEC"); + Base64.Decoder b64 = Base64.getDecoder(); + CertificateFactory kf = CertificateFactory.getInstance("X.509"); + + byte[] cert = b64.decode(ThroughputLatencyClient.pubKey); + InputStream certstream = new ByteArrayInputStream (cert); + + eng.initVerify(kf.generateCertificate(certstream)); + + } + eng.update(request); + if (!eng.verify(signature)) { + + System.out.println("Client sent invalid signature!"); + System.exit(0); + } + } + + } catch (NoSuchAlgorithmException | InvalidKeyException | SignatureException | CertificateException ex) { + ex.printStackTrace(); + System.exit(0); + } catch (NoSuchProviderException ex) { + ex.printStackTrace(); + System.exit(0); + } + + boolean readOnly = false; + + iterations++; + + if (msgCtx != null && msgCtx.getFirstInBatch() != null) { + + + readOnly = msgCtx.readOnly; + + msgCtx.getFirstInBatch().executedTime = System.nanoTime(); + + totalLatency.store(msgCtx.getFirstInBatch().executedTime - msgCtx.getFirstInBatch().receptionTime); + + if (readOnly == false) { + + consensusLatency.store(msgCtx.getFirstInBatch().decisionTime - msgCtx.getFirstInBatch().consensusStartTime); + long temp = msgCtx.getFirstInBatch().consensusStartTime - msgCtx.getFirstInBatch().receptionTime; + preConsLatency.store(temp > 0 ? temp : 0); + posConsLatency.store(msgCtx.getFirstInBatch().executedTime - msgCtx.getFirstInBatch().decisionTime); + proposeLatency.store(msgCtx.getFirstInBatch().writeSentTime - msgCtx.getFirstInBatch().consensusStartTime); + writeLatency.store(msgCtx.getFirstInBatch().acceptSentTime - msgCtx.getFirstInBatch().writeSentTime); + acceptLatency.store(msgCtx.getFirstInBatch().decisionTime - msgCtx.getFirstInBatch().acceptSentTime); + + } else { + + + consensusLatency.store(0); + preConsLatency.store(0); + posConsLatency.store(0); + proposeLatency.store(0); + writeLatency.store(0); + acceptLatency.store(0); + + + } + + } else { + + + consensusLatency.store(0); + preConsLatency.store(0); + posConsLatency.store(0); + proposeLatency.store(0); + writeLatency.store(0); + acceptLatency.store(0); + + + } + + float tp = -1; + if(iterations % interval == 0) { + if (context) System.out.println("--- (Context) iterations: "+ iterations + " // regency: " + msgCtx.getRegency() + " // consensus: " + msgCtx.getConsensusId() + " ---"); + + System.out.println("--- Measurements after "+ iterations+" ops ("+interval+" samples) ---"); + + tp = (float)(interval*1000/(float)(System.currentTimeMillis()-throughputMeasurementStartTime)); + + if (tp > maxTp) maxTp = tp; + + System.out.println("Throughput = " + tp +" operations/sec (Maximum observed: " + maxTp + " ops/sec)"); + + System.out.println("Total latency = " + totalLatency.getAverage(false) / 1000 + " (+/- "+ (long)totalLatency.getDP(false) / 1000 +") us "); + totalLatency.reset(); + System.out.println("Consensus latency = " + consensusLatency.getAverage(false) / 1000 + " (+/- "+ (long)consensusLatency.getDP(false) / 1000 +") us "); + consensusLatency.reset(); + System.out.println("Pre-consensus latency = " + preConsLatency.getAverage(false) / 1000 + " (+/- "+ (long)preConsLatency.getDP(false) / 1000 +") us "); + preConsLatency.reset(); + System.out.println("Pos-consensus latency = " + posConsLatency.getAverage(false) / 1000 + " (+/- "+ (long)posConsLatency.getDP(false) / 1000 +") us "); + posConsLatency.reset(); + System.out.println("Propose latency = " + proposeLatency.getAverage(false) / 1000 + " (+/- "+ (long)proposeLatency.getDP(false) / 1000 +") us "); + proposeLatency.reset(); + System.out.println("Write latency = " + writeLatency.getAverage(false) / 1000 + " (+/- "+ (long)writeLatency.getDP(false) / 1000 +") us "); + writeLatency.reset(); + System.out.println("Accept latency = " + acceptLatency.getAverage(false) / 1000 + " (+/- "+ (long)acceptLatency.getDP(false) / 1000 +") us "); + acceptLatency.reset(); + + System.out.println("Batch average size = " + batchSize.getAverage(false) + " (+/- "+ (long)batchSize.getDP(false) +") requests"); + batchSize.reset(); + + throughputMeasurementStartTime = System.currentTimeMillis(); + } + + return reply; + } +*/ public static void main(String[] args){ - if(args.length < 5) { - System.out.println("Usage: ... ThroughputLatencyServer "); + if(args.length < 6) { + System.out.println("Usage: ... ThroughputLatencyServer [rwd | rw]"); System.exit(-1); } @@ -180,8 +411,26 @@ public static void main(String[] args){ int replySize = Integer.parseInt(args[2]); int stateSize = Integer.parseInt(args[3]); boolean context = Boolean.parseBoolean(args[4]); + String signed = args[5]; + String write = args.length > 6 ? args[6] : ""; + + int s = 0; + + if (!signed.equalsIgnoreCase("nosig")) s++; + if (signed.equalsIgnoreCase("ecdsa")) s++; + + if (s == 2 && Security.getProvider("SunEC") == null) { + + System.out.println("Option 'ecdsa' requires SunEC provider to be available."); + System.exit(0); + } + + int w = 0; + + if (!write.equalsIgnoreCase("")) w++; + if (write.equalsIgnoreCase("rwd")) w++; - new ThroughputLatencyServer(processId,interval,replySize, stateSize, context); + new ThroughputLatencyServer(processId,interval,replySize, stateSize, context, s, w); } @Override diff --git a/src/bftsmart/reconfiguration/ClientViewController.java b/src/bftsmart/reconfiguration/ClientViewController.java index c7d458d2d..807f780e5 100644 --- a/src/bftsmart/reconfiguration/ClientViewController.java +++ b/src/bftsmart/reconfiguration/ClientViewController.java @@ -26,8 +26,8 @@ */ public class ClientViewController extends ViewController { - public ClientViewController(int procId, KeyLoader loader, Provider provider) { - super(procId, loader, provider); + public ClientViewController(int procId, KeyLoader loader) { + super(procId, loader); View cv = getViewStore().readView(); if(cv == null){ reconfigureTo(new View(0, getStaticConf().getInitialView(), @@ -37,8 +37,8 @@ public ClientViewController(int procId, KeyLoader loader, Provider provider) { } } - public ClientViewController(int procId, String configHome, KeyLoader loader, Provider provider) { - super(procId, configHome, loader, provider); + public ClientViewController(int procId, String configHome, KeyLoader loader) { + super(procId, configHome, loader); View cv = getViewStore().readView(); if(cv == null){ reconfigureTo(new View(0, getStaticConf().getInitialView(), diff --git a/src/bftsmart/reconfiguration/Reconfiguration.java b/src/bftsmart/reconfiguration/Reconfiguration.java index 15650a279..f696c1555 100644 --- a/src/bftsmart/reconfiguration/Reconfiguration.java +++ b/src/bftsmart/reconfiguration/Reconfiguration.java @@ -19,7 +19,6 @@ import bftsmart.tom.core.messages.TOMMessageType; import bftsmart.tom.util.KeyLoader; import bftsmart.tom.util.TOMUtil; -import java.security.Provider; /** * @@ -32,27 +31,28 @@ public class Reconfiguration { private int id; private KeyLoader keyLoader; - private Provider provider; private String configDir; - public Reconfiguration(int id, String configDir, KeyLoader loader, Provider provider) { - this.id = id; - + public Reconfiguration(int id, String configDir, KeyLoader loader) { + this.id = id; this.keyLoader = loader; - this.provider = provider; this.configDir = configDir; - //proxy = new ServiceProxy(id); - //request = new ReconfigureRequest(id); } public void connect(){ if(proxy == null){ - proxy = new ServiceProxy(id, configDir, null, null, keyLoader, provider); + proxy = new ServiceProxy(id, configDir, null, null, + keyLoader); } } - public void addServer(int id, String ip, int port){ - this.setReconfiguration(ServerViewController.ADD_SERVER, id + ":" + ip + ":" + port); + public void addServer(int id, String ip, int port, int portRR){ + this.setReconfiguration( + ServerViewController.ADD_SERVER, + id + ":" + + ip + ":" + + port + ":" + + portRR); } public void removeServer(int id){ @@ -61,11 +61,12 @@ public void removeServer(int id){ public void setF(int f){ - this.setReconfiguration(ServerViewController.CHANGE_F,String.valueOf(f)); + this.setReconfiguration(ServerViewController.CHANGE_F, String.valueOf(f)); } public void setReconfiguration(int prop, String value){ + System.out.println("setReconfiguration: Prop: " +prop+ ", Value: "+ value); if(request == null){ //request = new ReconfigureRequest(proxy.getViewManager().getStaticConf().getProcessId()); request = new ReconfigureRequest(id); @@ -74,8 +75,7 @@ public void setReconfiguration(int prop, String value){ } public ReconfigureReply execute(){ - byte[] signature = TOMUtil.signMessage(proxy.getViewManager().getStaticConf().getPrivateKey(), - request.toString().getBytes()); + byte[] signature = TOMUtil.signMessage(proxy.getViewManager().getStaticConf().getPrivateKey(), request.toString().getBytes()); request.setSignature(signature); byte[] reply = proxy.invoke(TOMUtil.getBytes(request), TOMMessageType.RECONFIG); request = null; diff --git a/src/bftsmart/reconfiguration/ReconfigurationTest.java b/src/bftsmart/reconfiguration/ReconfigurationTest.java index 0a710c6de..7731796af 100644 --- a/src/bftsmart/reconfiguration/ReconfigurationTest.java +++ b/src/bftsmart/reconfiguration/ReconfigurationTest.java @@ -38,7 +38,7 @@ public void run(int id){ ReconfigureReply r = (ReconfigureReply)TOMUtil.getObject(reply);*/ - Reconfiguration rec = new Reconfiguration(id, "", null, null); + Reconfiguration rec = new Reconfiguration(id, "", null); //rec.setReconfiguration(ReconfigurationManager.CHANGE_F,"1"); rec.setF(2); diff --git a/src/bftsmart/reconfiguration/ServerViewController.java b/src/bftsmart/reconfiguration/ServerViewController.java index b3c3610ca..e8bb1c01f 100644 --- a/src/bftsmart/reconfiguration/ServerViewController.java +++ b/src/bftsmart/reconfiguration/ServerViewController.java @@ -51,8 +51,8 @@ public class ServerViewController extends ViewController { private TOMLayer tomLayer; // protected View initialView; - public ServerViewController(int procId, KeyLoader loader, Provider provider) { - this(procId,"", loader, provider); + public ServerViewController(int procId, KeyLoader loader) { + this(procId,"", loader); /*super(procId); initialView = new View(0, getStaticConf().getInitialView(), getStaticConf().getF(), getInitAdddresses()); @@ -60,8 +60,8 @@ public ServerViewController(int procId, KeyLoader loader, Provider provider) { reconfigureTo(initialView);*/ } - public ServerViewController(int procId, String configHome, KeyLoader loader, Provider provider) { - super(procId, configHome, loader, provider); + public ServerViewController(int procId, String configHome, KeyLoader loader) { + super(procId, configHome, loader); View cv = getViewStore().readView(); if(cv == null){ @@ -109,42 +109,13 @@ public boolean hasUpdates() { public void enqueueUpdate(TOMMessage up) { ReconfigureRequest request = (ReconfigureRequest) TOMUtil.getObject(up.getContent()); - if (TOMUtil.verifySignature(getStaticConf().getPublicKey(request.getSender()), - request.toString().getBytes(), request.getSignature())) { - if (request.getSender() == getStaticConf().getTTPId()) { + if (request != null && request.getSender() == getStaticConf().getTTPId() + && TOMUtil.verifySignature(getStaticConf().getPublicKey(request.getSender()), + request.toString().getBytes(), request.getSignature())) { + //if (request.getSender() == getStaticConf().getTTPId()) { this.updates.add(up); - } else { - boolean add = true; - Iterator it = request.getProperties().keySet().iterator(); - while (it.hasNext()) { - int key = it.next(); - String value = request.getProperties().get(key); - if (key == ADD_SERVER) { - StringTokenizer str = new StringTokenizer(value, ":"); - if (str.countTokens() > 2) { - int id = Integer.parseInt(str.nextToken()); - if(id != request.getSender()){ - add = false; - } - }else{ - add = false; - } - } else if (key == REMOVE_SERVER) { - if (isCurrentViewMember(Integer.parseInt(value))) { - if(Integer.parseInt(value) != request.getSender()){ - add = false; - } - }else{ - add = false; - } - } else if (key == CHANGE_F) { - add = false; - } - } - if(add){ - this.updates.add(up); - } - } + } else { + logger.warn("Invalid reconfiguration from {}, discarding", up.getSender()); } } @@ -175,8 +146,8 @@ public byte[] executeUpdates(int cid) { jSet.add(id); String host = str.nextToken(); int port = Integer.valueOf(str.nextToken()); - this.getStaticConf().addHostInfo(id, host, port); - } + int portRR = Integer.valueOf(str.nextToken()); + this.getStaticConf().addHostInfo(id, host, port, portRR); } } } else if (key == REMOVE_SERVER) { if (isCurrentViewMember(Integer.parseInt(value))) { @@ -188,7 +159,6 @@ public byte[] executeUpdates(int cid) { } } - //ret = reconfigure(updates.get(i).getContent()); return reconfigure(jSetInfo, jSet, rSet, f, cid); } @@ -202,9 +172,6 @@ private boolean contains(int id, List list) { } private byte[] reconfigure(List jSetInfo, List jSet, List rSet, int f, int cid) { - //ReconfigureRequest request = (ReconfigureRequest) TOMUtil.getObject(req); - // Hashtable props = request.getProperties(); - // int f = Integer.valueOf(props.get(CHANGE_F)); lastJoinStet = new int[jSet.size()]; int[] nextV = new int[currentView.getN() + jSet.size() - rSet.size()]; int p = 0; @@ -292,7 +259,8 @@ public void processJoinResult(ReconfigureReply r) { this.lastJoinStet[i] = id; String host = str.nextToken(); int port = Integer.valueOf(str.nextToken()); - this.getStaticConf().addHostInfo(id, host, port); + int portRR = Integer.valueOf(str.nextToken()); + this.getStaticConf().addHostInfo(id, host, port, portRR); } } diff --git a/src/bftsmart/reconfiguration/VMServices.java b/src/bftsmart/reconfiguration/VMServices.java index c5ebb0444..453e5a01e 100644 --- a/src/bftsmart/reconfiguration/VMServices.java +++ b/src/bftsmart/reconfiguration/VMServices.java @@ -16,46 +16,62 @@ package bftsmart.reconfiguration; import bftsmart.tom.util.KeyLoader; -import java.security.Provider; /** - * - * @author Andre Nogueira + * This class is used by the trusted client to add and remove replicas from the group */ public class VMServices { private KeyLoader keyLoader; - private Provider provider; private String configDir; + /** + * Constructor. It adopts the default RSA key loader and default configuration path. + */ public VMServices() { // for the default keyloader and provider keyLoader = null; - provider = null; configDir = ""; } - public VMServices(KeyLoader keyLoader, Provider provider, String configDir) { + /** + * Constructor. + * + * @param keyLoader Key loader to use to fetch keys from disk + * @param configDir Configuration path + */ + public VMServices(KeyLoader keyLoader, String configDir) { this.keyLoader = keyLoader; - this.provider = provider; this.configDir = configDir; } - public void addServer(int id, String ipAddress, int port) { + /** + * Adds a new server to the group + * + * @param id ID of the server to be added (needs to match the value in config/hosts.config) + * @param ipAddress IP address of the server to be added (needs to match the value in config/hosts.config) + * @param port Port of the server to be added (needs to match the value in config/hosts.config) + */ + public void addServer(int id, String ipAddress, int port, int portRR) { - ViewManager viewManager = new ViewManager(configDir, keyLoader, provider); + ViewManager viewManager = new ViewManager(configDir, keyLoader); - viewManager.addServer(id, ipAddress,port); + viewManager.addServer(id, ipAddress, port, portRR); execute(viewManager); } + /** + * Removes a server from the group + * + * @param id ID of the server to be removed + */ public void removeServer (int id) { - ViewManager viewManager = new ViewManager(keyLoader, provider); + ViewManager viewManager = new ViewManager(keyLoader); viewManager.removeServer(id); diff --git a/src/bftsmart/reconfiguration/ViewController.java b/src/bftsmart/reconfiguration/ViewController.java index 2ed13eb08..079a05496 100644 --- a/src/bftsmart/reconfiguration/ViewController.java +++ b/src/bftsmart/reconfiguration/ViewController.java @@ -35,13 +35,13 @@ public class ViewController { private TOMConfiguration staticConf; private ViewStorage viewStore; - public ViewController(int procId, KeyLoader loader, Provider provider) { - this.staticConf = new TOMConfiguration(procId, loader, provider); + public ViewController(int procId, KeyLoader loader) { + this.staticConf = new TOMConfiguration(procId, loader); } - public ViewController(int procId, String configHome, KeyLoader loader, Provider provider) { - this.staticConf = new TOMConfiguration(procId, configHome, loader, provider); + public ViewController(int procId, String configHome, KeyLoader loader) { + this.staticConf = new TOMConfiguration(procId, configHome, loader); } @@ -51,7 +51,7 @@ public final ViewStorage getViewStore() { try { this.viewStore = (ViewStorage) Class.forName(className).newInstance(); } catch (Exception e) { - this.viewStore = new DefaultViewStorage(); + this.viewStore = new DefaultViewStorage(this.staticConf.getConfigHome()); } } diff --git a/src/bftsmart/reconfiguration/ViewManager.java b/src/bftsmart/reconfiguration/ViewManager.java index b7d306fbf..48f0f6450 100644 --- a/src/bftsmart/reconfiguration/ViewManager.java +++ b/src/bftsmart/reconfiguration/ViewManager.java @@ -22,16 +22,14 @@ import java.io.ObjectOutputStream; import java.util.LinkedList; import java.util.List; -import java.util.Scanner; import java.util.StringTokenizer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + import bftsmart.communication.server.ServerConnection; import bftsmart.reconfiguration.views.View; import bftsmart.tom.util.KeyLoader; -import java.security.Provider; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; /** * @@ -49,14 +47,14 @@ public class ViewManager { //in the system will execute the reconfiguration request private List addIds = new LinkedList(); - public ViewManager(KeyLoader loader, Provider provider) { - this("", loader, provider); + public ViewManager(KeyLoader loader) { + this("", loader); } - public ViewManager(String configHome, KeyLoader loader, Provider provider) { + public ViewManager(String configHome, KeyLoader loader) { this.id = loadID(configHome); - this.controller = new ServerViewController(id, configHome, loader, provider); - this.rec = new Reconfiguration(id, configHome, loader, provider); + this.controller = new ServerViewController(id, configHome, loader); + this.rec = new Reconfiguration(id, configHome, loader); } public void connect(){ @@ -95,9 +93,9 @@ private int loadID(String configHome) { } } - public void addServer(int id, String ip, int port) { - this.controller.getStaticConf().addHostInfo(id, ip, port); - rec.addServer(id, ip, port); + public void addServer(int id, String ip, int port, int portRR) { + this.controller.getStaticConf().addHostInfo(id, ip, port, portRR); + rec.addServer(id, ip, port, portRR); addIds.add(id); } @@ -121,8 +119,6 @@ public void executeUpdates() { sendResponse(addIds.toArray(new Integer[1]), msg); addIds.clear(); } - - } private ServerConnection getConnection(int remoteId) { @@ -141,17 +137,15 @@ public void sendResponse(Integer[] targets, VMMessage sm) { byte[] data = bOut.toByteArray(); for (Integer i : targets) { - //br.ufsc.das.tom.util.Logger.println("(ServersCommunicationLayer.send) Sending msg to replica "+i); try { if (i.intValue() != id) { - getConnection(i.intValue()).send(data, true); + getConnection(i.intValue()).send(data); } } catch (InterruptedException ex) { // ex.printStackTrace(); logger.error("Failed to send data to target", ex); } } - //br.ufsc.das.tom.util.Logger.println("(ServersCommunicationLayer.send) Finished sending messages to replicas"); } public void close() { diff --git a/src/bftsmart/reconfiguration/util/Configuration.java b/src/bftsmart/reconfiguration/util/Configuration.java index 0e362bd19..13e67282e 100644 --- a/src/bftsmart/reconfiguration/util/Configuration.java +++ b/src/bftsmart/reconfiguration/util/Configuration.java @@ -15,308 +15,330 @@ */ package bftsmart.reconfiguration.util; -import bftsmart.tom.util.KeyLoader; -import bftsmart.tom.util.TOMUtil; import java.io.BufferedReader; import java.io.FileReader; -import java.math.BigInteger; import java.net.InetSocketAddress; import java.security.PrivateKey; -import java.security.Provider; import java.security.PublicKey; -import java.security.Security; import java.util.HashMap; import java.util.Map; import java.util.StringTokenizer; -import org.bouncycastle.jce.provider.BouncyCastleProvider; -import org.slf4j.LoggerFactory; import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import bftsmart.tom.util.KeyLoader; +import bftsmart.tom.util.TOMUtil; public class Configuration { - - private Logger logger = LoggerFactory.getLogger(this.getClass()); - - protected int processId; - protected boolean channelsBlocking; - protected BigInteger DH_P; - protected BigInteger DH_G; - protected int autoConnectLimit; - protected Map configs; - protected HostsConfig hosts; - protected KeyLoader keyLoader; - protected Provider provider; - - - public static final String DEFAULT_HMAC = "HmacSHA512"; - public static final String DEFAULT_SECRETKEY = "PBEWithSHA1AndDES"; - public static final String DEFAULT_SIGNATURE = "SHA512withRSA"; - public static final String DEFAULT_HASH = "SHA-512"; - - private String hmacAlgorithm; - private String secretKeyAlgorithm; - private String signatureAlgorithm; - private String hashAlgorithm; - - protected static String configHome = ""; - - - protected static String hostsFileName = ""; - - protected boolean defaultKeys = false; - - public Configuration(int procId, KeyLoader loader, Provider prov){ - processId = procId; - keyLoader = loader; - provider = prov; - init(); - } - - public Configuration(int procId, String configHomeParam, KeyLoader loader, Provider prov){ - processId = procId; - configHome = configHomeParam; - keyLoader = loader; - provider = prov; - init(); - } - - protected void init(){ - try{ - hosts = new HostsConfig(configHome, hostsFileName); - - loadConfig(); - - String s = (String) configs.remove("system.autoconnect"); - if(s == null){ - autoConnectLimit = -1; - }else{ - autoConnectLimit = Integer.parseInt(s); - } - - s = (String) configs.remove("system.channels.blocking"); - if(s == null){ - channelsBlocking = false; - }else{ - channelsBlocking = (s.equalsIgnoreCase("true"))?true:false; - } - - s = (String) configs.remove("system.communication.hmacAlgorithm"); - if(s == null){ - hmacAlgorithm = DEFAULT_HMAC; - }else{ - hmacAlgorithm = s; - } - - s = (String) configs.remove("system.communication.secretKeyAlgorithm"); - if(s == null){ - secretKeyAlgorithm = DEFAULT_SECRETKEY; - }else{ - secretKeyAlgorithm = s; - } - - s = (String) configs.remove("system.communication.signatureAlgorithm"); - if(s == null){ - signatureAlgorithm = DEFAULT_SIGNATURE; - }else{ - signatureAlgorithm = s; - } - - s = (String) configs.remove("system.communication.hashAlgorithm"); - if(s == null){ - hashAlgorithm = DEFAULT_HASH; - }else{ - hashAlgorithm = s; - } - - s = (String) configs.remove("system.communication.defaultkeys"); - if(s == null){ - defaultKeys = false; - }else{ - defaultKeys = (s.equalsIgnoreCase("true"))?true:false; - } - - s = (String) configs.remove("system.diffie-hellman.p"); - if (s == null) { - String pHexString = "FFFFFFFF FFFFFFFF C90FDAA2 2168C234 C4C6628B 80DC1CD1" - + "29024E08 8A67CC74 020BBEA6 3B139B22 514A0879 8E3404DD" - + "EF9519B3 CD3A431B 302B0A6D F25F1437 4FE1356D 6D51C245" - + "E485B576 625E7EC6 F44C42E9 A637ED6B 0BFF5CB6 F406B7ED" - + "EE386BFB 5A899FA5 AE9F2411 7C4B1FE6 49286651 ECE65381" - + "FFFFFFFF FFFFFFFF"; - DH_P = new BigInteger(pHexString.replaceAll(" ", ""), 16); - } else { - DH_P = new BigInteger(s,16); - } - s = (String) configs.remove("system.diffie-hellman.g"); - if (s == null) { - DH_G = new BigInteger("2"); - } else { - DH_G = new BigInteger(s); - } - - if (keyLoader == null) keyLoader = new RSAKeyLoader(processId, TOMConfiguration.configHome, defaultKeys, signatureAlgorithm); - if (provider == null) provider = new BouncyCastleProvider(); - - TOMUtil.init(provider, hmacAlgorithm, secretKeyAlgorithm, keyLoader.getSignatureAlgorithm(), hashAlgorithm); - - }catch(Exception e){ - LoggerFactory.getLogger(this.getClass()).error("Wrong system.config file format."); - } - } - - public boolean useDefaultKeys() { - return defaultKeys; - } - - public final boolean isHostSetted(int id){ - if(hosts.getHost(id) == null){ - return false; - } - return true; - } - - - public final boolean useBlockingChannels(){ - return this.channelsBlocking; - } - - public final int getAutoConnectLimit(){ - return this.autoConnectLimit; - } - - public final BigInteger getDHP(){ - return DH_P; - } - - public final BigInteger getDHG(){ - return DH_G; - } - - public final String getHmacAlgorithm() { - return hmacAlgorithm; - } - - public final String getSecretKeyAlgorithm() { - return secretKeyAlgorithm; - } - - public final String getSignatureAlgorithm() { - return signatureAlgorithm; - } - - public final String getHashAlgorithm() { - return hashAlgorithm; - } - - public final String getProperty(String key){ - Object o = configs.get(key); - if( o != null){ - return o.toString(); - } - return null; - } - - public final Map getProperties(){ - return configs; - } - - public final InetSocketAddress getRemoteAddress(int id){ - return hosts.getRemoteAddress(id); - } - - - public final InetSocketAddress getServerToServerRemoteAddress(int id){ - return hosts.getServerToServerRemoteAddress(id); - } - - - public final InetSocketAddress getLocalAddress(int id){ - return hosts.getLocalAddress(id); - } - - public final String getHost(int id){ - return hosts.getHost(id); - } - - public final int getPort(int id){ - return hosts.getPort(id); - } - - public final int getServerToServerPort(int id){ - return hosts.getServerToServerPort(id); - } - - - public final int getProcessId(){ - return processId; - } - - public final void setProcessId(int processId){ - this.processId = processId; - } - - - public final void addHostInfo(int id, String host, int port){ - this.hosts.add(id,host,port); - } - - public PublicKey getPublicKey() { - try { - return keyLoader.loadPublicKey(); - } catch (Exception e) { - logger.error("Could not load public key",e); - return null; - } - - } - - public PublicKey getPublicKey(int id) { - try { - return keyLoader.loadPublicKey(id); - } catch (Exception e) { - logger.error("Could not load public key",e); - return null; - } - - } - - public PrivateKey getPrivateKey() { - try { - return keyLoader.loadPrivateKey(); - } catch (Exception e) { - logger.error("Could not load private key",e); - return null; - } - } - - public Provider getProvider() { - - return provider; - } - - private void loadConfig(){ - configs = new HashMap<>(); - try{ - if(configHome == null || configHome.equals("")){ - configHome="config"; - } - String sep = System.getProperty("file.separator"); - String path = configHome+sep+"system.config";; - FileReader fr = new FileReader(path); - BufferedReader rd = new BufferedReader(fr); - String line = null; - while((line = rd.readLine()) != null){ - if(!line.startsWith("#")){ - StringTokenizer str = new StringTokenizer(line,"="); - if(str.countTokens() > 1){ - configs.put(str.nextToken().trim(),str.nextToken().trim()); - } - } - } - fr.close(); - rd.close(); - }catch(Exception e){ - LoggerFactory.getLogger(this.getClass()).error("Could not load configuration",e); - } - } + + private Logger logger; + + protected int processId; + protected boolean channelsBlocking; + protected int autoConnectLimit; + protected Map configs; + protected HostsConfig hosts; + protected KeyLoader keyLoader; + + public static final String DEFAULT_SECRETKEY = "PBKDF2WithHmacSHA1"; + public static final String DEFAULT_SECRETKEY_PROVIDER = "SunJCE"; + + public static final String DEFAULT_SIGNATURE = "SHA512withECDSA"; + public static final String DEFAULT_SIGNATURE_PROVIDER = "BC"; + public static final String DEFAULT_KEYLOADER = "ECDSA"; + + public static final String DEFAULT_HASH = "SHA-512"; + public static final String DEFAULT_HASH_PROVIDER = "SUN"; + + + protected String secretKeyAlgorithm; + protected String secretKeyAlgorithmProvider; + + protected String signatureAlgorithm; + protected String signatureAlgorithmProvider; + + protected String hashAlgorithm; + protected String hashAlgorithmProvider; + + private String defaultKeyLoader; + + protected String configHome = ""; + + protected static String hostsFileName = ""; + + protected boolean defaultKeys = false; + + public Configuration(int procId, KeyLoader loader) { + logger = LoggerFactory.getLogger(this.getClass()); + processId = procId; + keyLoader = loader; + init(); + } + + public Configuration(int procId, String configHomeParam, KeyLoader loader) { + logger = LoggerFactory.getLogger(this.getClass()); + processId = procId; + configHome = configHomeParam; + keyLoader = loader; + init(); + } + + protected void init() { + logger = LoggerFactory.getLogger(this.getClass()); + try { + hosts = new HostsConfig(configHome, hostsFileName); + + loadConfig(); + + String s = (String) configs.remove("system.autoconnect"); + if (s == null) { + autoConnectLimit = -1; + } else { + autoConnectLimit = Integer.parseInt(s); + } + + s = (String) configs.remove("system.channels.blocking"); + if (s == null) { + channelsBlocking = false; + } else { + channelsBlocking = (s.equalsIgnoreCase("true")) ? true : false; + } + + s = (String) configs.remove("system.communication.secretKeyAlgorithm"); + if (s == null) { + secretKeyAlgorithm = DEFAULT_SECRETKEY; + } else { + secretKeyAlgorithm = s; + } + + s = (String) configs.remove("system.communication.signatureAlgorithm"); + if (s == null) { + signatureAlgorithm = DEFAULT_SIGNATURE; + } else { + signatureAlgorithm = s; + } + + s = (String) configs.remove("system.communication.hashAlgorithm"); + if (s == null) { + hashAlgorithm = DEFAULT_HASH; + } else { + hashAlgorithm = s; + } + + s = (String) configs.remove("system.communication.secretKeyAlgorithmProvider"); + if (s == null) { + secretKeyAlgorithmProvider = DEFAULT_SECRETKEY_PROVIDER; + } else { + secretKeyAlgorithmProvider = s; + } + + s = (String) configs.remove("system.communication.signatureAlgorithmProvider"); + if (s == null) { + signatureAlgorithmProvider = DEFAULT_SIGNATURE_PROVIDER; + } else { + signatureAlgorithmProvider = s; + } + + s = (String) configs.remove("system.communication.hashAlgorithmProvider"); + if (s == null) { + hashAlgorithmProvider = DEFAULT_HASH_PROVIDER; + } else { + hashAlgorithmProvider = s; + } + + s = (String) configs.remove("system.communication.defaultKeyLoader"); + if (s == null) { + defaultKeyLoader = DEFAULT_KEYLOADER; + } else { + defaultKeyLoader = s; + } + + s = (String) configs.remove("system.communication.defaultkeys"); + if (s == null) { + defaultKeys = false; + } else { + defaultKeys = (s.equalsIgnoreCase("true")) ? true : false; + } + + if (keyLoader == null) { + switch (defaultKeyLoader) { + case "RSA": + keyLoader = new RSAKeyLoader(processId, configHome, defaultKeys, signatureAlgorithm); + break; + case "ECDSA": + keyLoader = new ECDSAKeyLoader(processId, configHome, defaultKeys, signatureAlgorithm); + break; + case "SunEC": + keyLoader = new SunECKeyLoader(processId, configHome, defaultKeys, signatureAlgorithm); + break; + default: + keyLoader = new ECDSAKeyLoader(processId, configHome, defaultKeys, signatureAlgorithm); + break; + } + } + + TOMUtil.init( + secretKeyAlgorithm, + keyLoader.getSignatureAlgorithm(), + hashAlgorithm, + secretKeyAlgorithmProvider, + signatureAlgorithmProvider, + hashAlgorithmProvider); + + } catch (Exception e) { + LoggerFactory.getLogger(this.getClass()).error("Wrong system.config file format."); + } + } + + public String getConfigHome() { + return configHome; + } + + public boolean useDefaultKeys() { + return defaultKeys; + } + + public final boolean isHostSetted(int id) { + if (hosts.getHost(id) == null) { + return false; + } + return true; + } + + public final boolean useBlockingChannels() { + return this.channelsBlocking; + } + + public final int getAutoConnectLimit() { + return this.autoConnectLimit; + } + + public final String getSecretKeyAlgorithm() { + return secretKeyAlgorithm; + } + + public final String getSignatureAlgorithm() { + return signatureAlgorithm; + } + + public final String getHashAlgorithm() { + return hashAlgorithm; + } + + public final String getSecretKeyAlgorithmProvider() { + return secretKeyAlgorithmProvider; + } + + public final String getSignatureAlgorithmProvider() { + return signatureAlgorithmProvider; + } + + public final String getHashAlgorithmProvider() { + return hashAlgorithmProvider; + } + + public final String getProperty(String key) { + Object o = configs.get(key); + if (o != null) { + return o.toString(); + } + return null; + } + + public final Map getProperties() { + return configs; + } + + public final InetSocketAddress getRemoteAddress(int id) { + return hosts.getRemoteAddress(id); + } + + public final InetSocketAddress getServerToServerRemoteAddress(int id) { + return hosts.getServerToServerRemoteAddress(id); + } + + public final InetSocketAddress getLocalAddress(int id) { + return hosts.getLocalAddress(id); + } + + public final String getHost(int id) { + return hosts.getHost(id); + } + + public final int getPort(int id) { + return hosts.getPort(id); + } + + public final int getServerToServerPort(int id) { + return hosts.getServerToServerPort(id); + } + + public final int getProcessId() { + return processId; + } + + public final void setProcessId(int processId) { + this.processId = processId; + } + + public final void addHostInfo(int id, String host, int port, int portRR) { + this.hosts.add(id, host, port, portRR); + } + + public PublicKey getPublicKey() { + try { + return keyLoader.loadPublicKey(); + } catch (Exception e) { + logger.error("Could not load public key", e); + return null; + } + + } + + public PublicKey getPublicKey(int id) { + try { + return keyLoader.loadPublicKey(id); + } catch (Exception e) { + logger.error("Could not load public key", e); + return null; + } + + } + + public PrivateKey getPrivateKey() { + try { + return keyLoader.loadPrivateKey(); + } catch (Exception e) { + logger.error("Could not load private key", e); + return null; + } + } + + private void loadConfig() { + configs = new HashMap<>(); + try { + if (configHome == null || configHome.equals("")) { + configHome = "config"; + } + String sep = System.getProperty("file.separator"); + String path = configHome + sep + "system.config"; + ; + FileReader fr = new FileReader(path); + BufferedReader rd = new BufferedReader(fr); + String line = null; + while ((line = rd.readLine()) != null) { + if (!line.startsWith("#")) { + StringTokenizer str = new StringTokenizer(line, "="); + if (str.countTokens() > 1) { + configs.put(str.nextToken().trim(), str.nextToken().trim()); + } + } + } + fr.close(); + rd.close(); + } catch (Exception e) { + LoggerFactory.getLogger(this.getClass()).error("Could not load configuration", e); + } + } } diff --git a/src/bftsmart/reconfiguration/util/DefaultVMServices.java b/src/bftsmart/reconfiguration/util/DefaultVMServices.java index cbecd6d42..38f42f34c 100644 --- a/src/bftsmart/reconfiguration/util/DefaultVMServices.java +++ b/src/bftsmart/reconfiguration/util/DefaultVMServices.java @@ -6,8 +6,6 @@ package bftsmart.reconfiguration.util; import bftsmart.reconfiguration.VMServices; -import bftsmart.tom.util.KeyLoader; -import java.security.Provider; /** * @@ -31,8 +29,9 @@ public static void main(String[] args) throws InterruptedException { int smartId = Integer.parseInt(args[0]); String ipAddress = args[1]; int port = Integer.parseInt(args[2]); + int portRR = Integer.parseInt(args[3]); - (new DefaultVMServices()).addServer(smartId, ipAddress, port); + (new DefaultVMServices()).addServer(smartId, ipAddress, port, portRR); }else{ System.out.println("Usage: java -jar TppServices [ip address] [port]"); diff --git a/src/bftsmart/reconfiguration/util/ECDSAKeyLoader.java b/src/bftsmart/reconfiguration/util/ECDSAKeyLoader.java new file mode 100644 index 000000000..d36c52d8d --- /dev/null +++ b/src/bftsmart/reconfiguration/util/ECDSAKeyLoader.java @@ -0,0 +1,223 @@ +/** +Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, Tulio A. Ribeiro, and the authors indicated in the @author tags + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ +package bftsmart.reconfiguration.util; + +import java.io.BufferedReader; +import java.io.FileReader; +import java.io.IOException; +import java.security.KeyFactory; +import java.security.NoSuchAlgorithmException; +import java.security.NoSuchProviderException; +import java.security.PrivateKey; +import java.security.PublicKey; +import java.security.cert.CertificateException; +import java.security.spec.EncodedKeySpec; +import java.security.spec.InvalidKeySpecException; +import java.security.spec.PKCS8EncodedKeySpec; +import java.security.spec.X509EncodedKeySpec; + +import org.apache.commons.codec.binary.Base64; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import bftsmart.tom.util.KeyLoader; + +/** + * Used to load JCA public and private keys from conf/keys/publickey and + * conf/keys/privatekey + */ +public class ECDSAKeyLoader implements KeyLoader { + + private Logger logger = LoggerFactory.getLogger(this.getClass()); + private String path; + private int id; + private PrivateKey privateKey = null; + + private String sigAlgorithm; + + private boolean defaultKeys; + + //Bouncy Castle | 256 key size + private static final String PRIVATE_KEY = "MIGTAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBHkwdwIBAQQgMMpfKtHS5ZlgHDPj3TG41Y0t5r9NIzx7p4YPZxn5gBmgCgYIKoZIzj0DAQehRANCAAQkD2DTG37xnxtcMMLJMiUCyObUdVJE+rMM9WQ1Z3sjtIZchN8Xefr02Ag+giXGLej862qu3v4/fy6UGJNAHNx3"; + private static final String PUBLIC_KEY = "MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEJA9g0xt+8Z8bXDDCyTIlAsjm1HVSRPqzDPVkNWd7I7SGXITfF3n69NgIPoIlxi3o/Otqrt7+P38ulBiTQBzcdw=="; + + //Bouncy Castle | 384 key size + //private static final String PRIVATE_KEY = "MIG/AgEAMBAGByqGSM49AgEGBSuBBAAiBIGnMIGkAgEBBDAgJ7PlzZFc8/rsMTODSsQgenL2+WDxGjohmwGSLe8wp0+dYX8x2M2CidHdjzQu41qgBwYFK4EEACKhZANiAASFk9m7oKkuGqHg+BDMo51XbDnvIahxmfcagDgOvU/plgjpyuoY74eJit2LlLhLyKzVq0Rmpr1dZAuQWhlZfmlvdv0RhScaSaOiea54VdO8ZhdHzHHDwZQkJiSJQAuWOv0="; + //private static final String PUBLIC_KEY = "MHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEhZPZu6CpLhqh4PgQzKOdV2w57yGocZn3GoA4Dr1P6ZYI6crqGO+HiYrdi5S4S8is1atEZqa9XWQLkFoZWX5pb3b9EYUnGkmjonmueFXTvGYXR8xxw8GUJCYkiUALljr9"; + + //Bouncy Castle | 521 key size + //private static final String PRIVATE_KEY = "MIH3AgEAMBAGByqGSM49AgEGBSuBBAAjBIHfMIHcAgEBBEIBlz6dy43Dp2XHkJzP00oY4japCVdVjYqUZdmDJwTnNfPCmiBA362OO8XgHzkSoz11W/YhXN3NNyBZg0gWNC1E4ZmgBwYFK4EEACOhgYkDgYYABAH0rMKko/e9Wp8f0G01SCbQaRBkZ/9PvxBKG3GbFUFeR5TiCf1GH8UNLHn5q6+ayD9RfhtOuSj2JuLKzZwAFNo12QAPXa/COqKxdwzoLnUcc81i1I/NEsgVp4eqHjs4UPzP9mvWE+D+XqXAqEU8cK+CMA9IXvdIrUU/szSvkhWT5nw0EA=="; + //private static final String PUBLIC_KEY = "MIGbMBAGByqGSM49AgEGBSuBBAAjA4GGAAQB9KzCpKP3vVqfH9BtNUgm0GkQZGf/T78QShtxmxVBXkeU4gn9Rh/FDSx5+auvmsg/UX4bTrko9ibiys2cABTaNdkAD12vwjqisXcM6C51HHPNYtSPzRLIFaeHqh47OFD8z/Zr1hPg/l6lwKhFPHCvgjAPSF73SK1FP7M0r5IVk+Z8NBA="; + + /** Creates a new instance of ECDSAKeyLoader */ + + public ECDSAKeyLoader(int id, String configHome, boolean defaultKeys, String sigAlgorithm) { + this.id = id; + this.defaultKeys = defaultKeys; + this.sigAlgorithm = sigAlgorithm; + + if (configHome.equals("")) { + path = "config" + System.getProperty("file.separator") + "keysECDSA" + + System.getProperty("file.separator"); + } else { + path = configHome + System.getProperty("file.separator") + "keysECDSA" + + System.getProperty("file.separator"); + } + + } + + /** + * Loads the public key of some processes from configuration files + * + * @return the PublicKey loaded from config/keys/publickey + * @throws Exception + * problems reading or parsing the key + */ + public PublicKey loadPublicKey(int id) + throws IOException, NoSuchAlgorithmException, InvalidKeySpecException, CertificateException { + + if (defaultKeys) { + //logger.debug("Loading default PublicKey, id: {}", id); + try { + return getPublicKeyFromString(PUBLIC_KEY); + } catch (NoSuchProviderException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + } + + FileReader f = new FileReader(path + "publickey" + id); + BufferedReader r = new BufferedReader(f); + String tmp = ""; + String key = ""; + while ((tmp = r.readLine()) != null) { + key = key + tmp; + } + f.close(); + r.close(); + PublicKey ret = null; + logger.debug("Loading PublicKey from file, id: {}", id); + try { + ret = getPublicKeyFromString(key); + } catch (NoSuchProviderException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + logger.trace("ID: {}, PublicKey Format: {}, PublicKey Algorithm: {} ", + new Object[] { id, ret.getFormat(), ret.getAlgorithm() }); + return ret; + } + + public PublicKey loadPublicKey() + throws IOException, NoSuchAlgorithmException, InvalidKeySpecException, CertificateException { + + if (defaultKeys) { + try { + return getPublicKeyFromString(PUBLIC_KEY); + } catch (NoSuchProviderException e) { + e.printStackTrace(); + } + } + logger.debug("Loading PublicKey from file, this.id: {}", this.id); + + FileReader f = new FileReader(path + "publickey" + this.id); + BufferedReader r = new BufferedReader(f); + String tmp = ""; + String key = ""; + while ((tmp = r.readLine()) != null) { + key = key + tmp; + } + f.close(); + r.close(); + PublicKey ret = null; + try { + ret = getPublicKeyFromString(PUBLIC_KEY); + } catch (NoSuchProviderException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + logger.trace("ID: {}, PublicKey Format: {}, PublicKey Algorithm: {} ", + new Object[] { this.id, ret.getFormat(), ret.getAlgorithm() }); + return ret; + + } + + /** + * Loads the private key of this process + * + * @return the PrivateKey loaded from config/keys/publickey + * @throws Exception + * problems reading or parsing the key + */ + public PrivateKey loadPrivateKey() throws IOException, NoSuchAlgorithmException, InvalidKeySpecException { + + if (defaultKeys) { + //logger.info("Loading private key..."); + try { + return getPrivateKeyFromString(PRIVATE_KEY); + } catch (NoSuchProviderException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + } + + if (privateKey == null) { + FileReader f = new FileReader(path + "privatekey" + this.id); + BufferedReader r = new BufferedReader(f); + String tmp = ""; + String key = ""; + while ((tmp = r.readLine()) != null) { + key = key + tmp; + } + f.close(); + r.close(); + logger.debug("Loading first time PrivateKey from file, this.id: {}, \nKey:{}", this.id, key); + try { + privateKey = getPrivateKeyFromString(key); + logger.trace("PrivateKey loaded for this.id: {}, PrivateKey Format: {}, PrivateKey Algorithm: {} ", + new Object[] { this.id, privateKey.getFormat(), privateKey.getAlgorithm() }); + } catch (NoSuchProviderException e) { + e.printStackTrace(); + } + } + logger.trace("Returning previous stored PrivateKey from file, this.id: {}", this.id); + return privateKey; + } + + // utility methods for going from string to public/private key + private PrivateKey getPrivateKeyFromString(String key) + throws NoSuchProviderException, NoSuchAlgorithmException, InvalidKeySpecException { + + KeyFactory keyFactory = KeyFactory.getInstance("ECDSA", "BC"); + EncodedKeySpec privateKeySpec = new PKCS8EncodedKeySpec(Base64.decodeBase64(key)); + privateKey = keyFactory.generatePrivate(privateKeySpec); + return privateKey; + } + + private PublicKey getPublicKeyFromString(String key) + throws NoSuchProviderException, NoSuchAlgorithmException, InvalidKeySpecException, NoSuchProviderException { + KeyFactory keyFactory = KeyFactory.getInstance("ECDSA", "BC"); + X509EncodedKeySpec pubKeySpec = new X509EncodedKeySpec(Base64.decodeBase64(key)); + PublicKey publicKey = keyFactory.generatePublic(pubKeySpec); + return publicKey; + } + + @Override + public String getSignatureAlgorithm() { + return this.sigAlgorithm; + } + +} diff --git a/src/bftsmart/reconfiguration/util/HostsConfig.java b/src/bftsmart/reconfiguration/util/HostsConfig.java index 5ef0791cc..550d82112 100644 --- a/src/bftsmart/reconfiguration/util/HostsConfig.java +++ b/src/bftsmart/reconfiguration/util/HostsConfig.java @@ -21,6 +21,7 @@ import java.util.Iterator; import java.util.Set; import java.util.StringTokenizer; + import org.slf4j.LoggerFactory; public class HostsConfig { @@ -54,11 +55,12 @@ private void loadConfig(String configHome, String fileName){ while((line = rd.readLine()) != null){ if(!line.startsWith("#")){ StringTokenizer str = new StringTokenizer(line," "); - if(str.countTokens() > 2){ + if(str.countTokens() == 4){ int id = Integer.valueOf(str.nextToken()); String host = str.nextToken(); int port = Integer.valueOf(str.nextToken()); - this.servers.put(id, new Config(id,host,port)); + int portRR = Integer.valueOf(str.nextToken()); + this.servers.put(id, new Config(id, host, port, portRR)); } } } @@ -69,9 +71,9 @@ private void loadConfig(String configHome, String fileName){ } } - public void add(int id, String host, int port){ + public void add(int id, String host, int port, int portRR){ if(this.servers.get(id) == null){ - this.servers.put(id, new Config(id,host,port)); + this.servers.put(id, new Config(id, host, port, portRR)); } } @@ -91,7 +93,7 @@ public InetSocketAddress getRemoteAddress(int id){ public InetSocketAddress getServerToServerRemoteAddress(int id){ Config c = (Config) this.servers.get(id); if(c != null){ - return new InetSocketAddress(c.host,c.port+1); + return new InetSocketAddress(c.host,c.portRR); } return null; } @@ -105,14 +107,15 @@ public int getPort(int id){ return -1; } - public int getServerToServerPort(int id){ + public int getServerToServerPort(int id){ Config c = (Config) this.servers.get(id); if(c != null){ - return c.port+1; + return c.portRR; } return -1; } + public int[] getHostsIds(){ @@ -156,11 +159,13 @@ public class Config{ public int id; public String host; public int port; + public int portRR; - public Config(int id, String host, int port){ + public Config(int id, String host, int port, int portRR){ this.id = id; this.host = host; this.port = port; + this.portRR = portRR; } } } diff --git a/src/bftsmart/reconfiguration/util/RSAKeyLoader.java b/src/bftsmart/reconfiguration/util/RSAKeyLoader.java index dd69c920e..8715f2fa0 100644 --- a/src/bftsmart/reconfiguration/util/RSAKeyLoader.java +++ b/src/bftsmart/reconfiguration/util/RSAKeyLoader.java @@ -28,6 +28,8 @@ import java.security.spec.InvalidKeySpecException; import java.security.spec.PKCS8EncodedKeySpec; import java.security.spec.X509EncodedKeySpec; +import java.util.HashMap; +import java.util.Map; import org.apache.commons.codec.binary.Base64; /** @@ -41,24 +43,31 @@ public class RSAKeyLoader implements KeyLoader { private PrivateKey priKey; private String sigAlgorithm; - - private static String DEFAULT_UKEY = "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCwuoTWbSFDnVjohwdZftoAwv3oCxUPnUiiNNH9\n" + - "\npXryEW8kSFRGVJ7zJCwxJnt3YZGnpPGxnC3hAI4XkG26hO7+TxkgaYmv5GbamL946uZISxv0aNX3\n" + - "\nYbaOf//MC6F8tShFfCnpWlj68FYulM5dC2OOOHaUJfofQhmXfsaEWU251wIDAQAB"; - - - private static String DEFAULT_PKEY = "MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBALC6hNZtIUOdWOiHB1l+2gDC/egL\n" + - "\nFQ+dSKI00f2levIRbyRIVEZUnvMkLDEme3dhkaek8bGcLeEAjheQbbqE7v5PGSBpia/kZtqYv3jq\n" + - "\n5khLG/Ro1fdhto5//8wLoXy1KEV8KelaWPrwVi6Uzl0LY444dpQl+h9CGZd+xoRZTbnXAgMBAAEC\n" + - "\ngYAJaUVdrd4RnbV4XIh1qZ2uYLPowX5ToIqXqLxuB3vunCMRCZEDVcpJJGn+DBCTIO0CwnPkg26m\n" + - "\nBsOKWbSeNCoN5gOi5yd6Poe0D40ZmvHP1hMCQ9LYhwjLB3Aa+Cl5gYL074Qe/eJFqJaYjZApkeJU\n" + - "\nAy1HkXhM5OBW9grrXxg6YQJBAPTIni5fG5f2SYutkR2pUydZP4haKVabRkZr8nSHuClGDE2HzbNQ\n" + - "\njb17z5rRVxJCKMLb2HiPg7ZsUgGK/J1ri78CQQC405h45rL1mCIyZQCXcK/nQZTVi8UaaelKN/ub\n" + - "\nLQKtTGenJao/zoL+m39i+gGRkHWiG6HNaGFdOkRJmeeH+rfpAkEAn0fwDjKbDP4ZC0fM1uU4k7Ey\n" + - "\nczJgFdgCGY7ifMtXnZvUI5sL0fPH15W6GH7BzsK4LVvK92BDj6/aiOB80p6JlwJASjL4NSE4mwv2\n" + - "\nPpD5ydI9a/OSEqDIAjCerWMIKWXKe1P/EMU4MeFwCVLXsx523r9F2kyJinLrE4g+veWBY7+tcQJB\n" + - "\nAKCTm3tbbwLJnnPN46mAgrYb5+LFOmHyNtEDgjxEVrzpQaCChZici2YGY1jTBjb/De4jii8RXllA\n" + - "\ntUhBEsqyXDA="; + + private Map pubKeys; + + private static String DEFAULT_UKEY = "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAokZC75w2IQLEyAgCpQqCDH3keTdHq+3lFOZJ"+ + "PbAev4zq73umOB3bFdSVu0OpbTwV7Mo7CHGTrtB4oi/REvgL6xwL/DKJ7Y2/cAQ91l4ApgmtyX6d0ESsVWZzCg57zjaiwHzzVN57R8q4/h3Cc"+ + "UxjDmCQtC9F4W83wm/sFvaTBovbkVQK5y2wBiQ3m+nFA9YWz+dgZy7wh4NJNbvnMpfhTBs73P64De6i2D/v2bjNJoke1mdSTM2+K9aSpwKBEe"+ + "dtI/mkQqQvA/eCAPNNDidXAVCewfHONpRu4wc/ovjPG+6AlrqRSEYy+GtAndgyPFc8L+VXAMdAyIe8109gTz4+lwIDAQAB"; + + + private static String DEFAULT_PKEY = "MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCiRkLvnDYhAsTICAKlCoIMfeR5N0e"+ + "r7eUU5kk9sB6/jOrve6Y4HdsV1JW7Q6ltPBXsyjsIcZOu0HiiL9ES+AvrHAv8Montjb9wBD3WXgCmCa3Jfp3QRKxVZnMKDnvONqLAfPNU3nt"+ + "Hyrj+HcJxTGMOYJC0L0XhbzfCb+wW9pMGi9uRVArnLbAGJDeb6cUD1hbP52BnLvCHg0k1u+cyl+FMGzvc/rgN7qLYP+/ZuM0miR7WZ1JMzb4"+ + "r1pKnAoER520j+aRCpC8D94IA800OJ1cBUJ7B8c42lG7jBz+i+M8b7oCWupFIRjL4a0Cd2DI8Vzwv5VcAx0DIh7zXT2BPPj6XAgMBAAECggE"+ + "AWzGdIEbrbHW/3KITymgzWY3OPgjA8HAK00nvUwmM3hz1UoxshxDQNF55nvmRV1/y+lVUA5TAZ/ekM5Enr79SA1iJq7tDJAsK0Iqxray6NJU"+ + "v4xKS4Z4WMxAWCkrFbMfrgr01ijZVlazpdXWH9l/1Mvk1mO0QGnaEIXMAfI8pZP2CXL3oEIv+uhbPytdkwlCFtCgwLpteTbmX806i5R2NaeM"+ + "S8JwGpI8bSL143Lxl1nR0w7X9+5LyXEFuOJRdy/qsso1/fqxFG/CXg3GCTJwWGsFWS2jEF0NVRAj/OOVlNp2smbvTRRXN+eRjAbc06R4aVlW"+ + "9GwkMq0xaX4DIzVhWwQKBgQDZp+dRTqLXHkvPrqhEQOrrFS2mKmjnD1vFn1hBCGcezRfVcdY2zPumZvmFvUvdcHB+sOo7UHCTzKLeu61eI26"+ + "qDmkSuATiyZuwp+rzjRzq9+lzdyq1ikP5XiCg/W5u0fXLFIbQ05o+T6Mp0DDXllKh0tKRFTP18FP3qcNEB3GwcQKBgQC+3LPo+EFXECZMTnZ"+ + "YnYOd6c+0KCWFgbgtbqsBUe6uZe5jiwrCq8ypAWcZpYBlEEnX0lZVfavSZLx+CUJE6zcnuV4B9qPjCEJMS1UTMDo2DPAbiUlNo29i/hgZ30U"+ + "hhcxeNcDxyV4LCwzeVXjSGoj6lpH912ckgE76aJf+izbjhwKBgADGGQtv9SJAqFJXs59yf6NQUvY5RmAz7MaaF207w2oXnpMSsYlGV0qzKQX"+ + "xs1hZMv0wUdTeJ1hPLPEPx8EC7TzLilIXt2S7BMOvBSXShZzMPtc7QDqfADjdvc30uLTKKE6NhyEs72pzTAg1Bkdt8GNE5ZzAb4vbS+EgiGI"+ + "wiJXxAoGBAI7kGwl6+ygcXh/Yyj9zxbru9mKRf+3g/St+ZCZ72a7Vf1ElIqw2BOYut6p6vpJrTG14+svMZ1v/sSLG+ccxNjzWSaw8o9vwLfq"+ + "Cl7Hi9GHM8+IZuTiX+Gdrhk2wW0hKrIOHyOj78h1ga4T1Bpx94zmAitI5du3b5cURk2Gthi13AoGAYj6ygbie5vGSEi68/6dB7whHLsk2aqL"+ + "9/8qe5CDGCPy43YhDXpHrJokHQ4xh3HR9QZuoEqV9oOYREVGjXYx4P1nUqKNMrbzXoypPczx+hNO22rJcu/flMmXGeC9q7tBbnuT6c8vKo3R"+ + "Lvwp7KA+fDYCMsfbeBa66hOVweG262hM="; private boolean defaultKeys; @@ -70,12 +79,14 @@ public RSAKeyLoader(int id, String configHome, boolean defaultKeys, String sigAl this.sigAlgorithm = sigAlgorithm; if (configHome.equals("")) { - path = "config" + System.getProperty("file.separator") + "keys" + + path = "config" + System.getProperty("file.separator") + "keysRSA" + System.getProperty("file.separator"); } else { - path = configHome + System.getProperty("file.separator") + "keys" + + path = configHome + System.getProperty("file.separator") + "keysRSA" + System.getProperty("file.separator"); } + + pubKeys = new HashMap<>(); } /** @@ -89,17 +100,25 @@ public PublicKey loadPublicKey(int id) throws IOException, NoSuchAlgorithmExcept if (defaultKeys) { return getPublicKeyFromString(RSAKeyLoader.DEFAULT_UKEY); } - - FileReader f = new FileReader(path + "publickey" + id); - BufferedReader r = new BufferedReader(f); - String tmp = ""; - String key = ""; - while ((tmp = r.readLine()) != null) { - key = key + tmp; + + PublicKey ret = pubKeys.get(id); + + if (ret == null) { + + FileReader f = new FileReader(path + "publickey" + id); + BufferedReader r = new BufferedReader(f); + String tmp = ""; + String key = ""; + while ((tmp = r.readLine()) != null) { + key = key + tmp; + } + f.close(); + r.close(); + ret = getPublicKeyFromString(key); + + pubKeys.put(id, ret); } - f.close(); - r.close(); - PublicKey ret = getPublicKeyFromString(key); + return ret; } @@ -108,17 +127,25 @@ public PublicKey loadPublicKey() throws IOException, NoSuchAlgorithmException, I if (defaultKeys) { return getPublicKeyFromString(RSAKeyLoader.DEFAULT_UKEY); } - - FileReader f = new FileReader(path + "publickey" + this.id); - BufferedReader r = new BufferedReader(f); - String tmp = ""; - String key = ""; - while ((tmp = r.readLine()) != null) { - key = key + tmp; + + PublicKey ret = pubKeys.get(this.id); + + if (ret == null) { + + FileReader f = new FileReader(path + "publickey" + this.id); + BufferedReader r = new BufferedReader(f); + String tmp = ""; + String key = ""; + while ((tmp = r.readLine()) != null) { + key = key + tmp; + } + f.close(); + r.close(); + ret = getPublicKeyFromString(key); + + pubKeys.put(this.id, ret); } - f.close(); - r.close(); - PublicKey ret = getPublicKeyFromString(key); + return ret; } diff --git a/src/bftsmart/reconfiguration/util/SunECKeyLoader.java b/src/bftsmart/reconfiguration/util/SunECKeyLoader.java new file mode 100644 index 000000000..bb95e39eb --- /dev/null +++ b/src/bftsmart/reconfiguration/util/SunECKeyLoader.java @@ -0,0 +1,248 @@ +/** +Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, Tulio A. Ribeiro, and the authors indicated in the @author tags + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ +package bftsmart.reconfiguration.util; + +import java.io.BufferedReader; +import java.io.FileReader; +import java.io.IOException; +import java.security.KeyFactory; +import java.security.NoSuchAlgorithmException; +import java.security.NoSuchProviderException; +import java.security.PrivateKey; +import java.security.PublicKey; +import java.security.Security; +import java.security.cert.CertificateException; +import java.security.interfaces.ECPublicKey; +import java.security.spec.ECPublicKeySpec; +import java.security.spec.EncodedKeySpec; +import java.security.spec.InvalidKeySpecException; +import java.security.spec.PKCS8EncodedKeySpec; +import java.security.spec.X509EncodedKeySpec; + +import org.apache.commons.codec.binary.Base64; +import org.bouncycastle.jce.ECNamedCurveTable; +import org.bouncycastle.jce.ECPointUtil; +import org.bouncycastle.jce.provider.BouncyCastleProvider; +import org.bouncycastle.jce.spec.ECNamedCurveParameterSpec; +import org.bouncycastle.jce.spec.ECNamedCurveSpec; +import org.bouncycastle.math.ec.ECPoint; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import bftsmart.tom.util.KeyLoader; + +/** + * Used to load JCA public and private keys from conf/keys/publickey and + * conf/keys/privatekey + */ +public class SunECKeyLoader implements KeyLoader { + + private Logger logger = LoggerFactory.getLogger(this.getClass()); + private String path; + private int id; + private PrivateKey privateKey = null; + + private String sigAlgorithm; + + private boolean defaultKeys; + + //SunEC secp256r1 + private static final String PRIVATE_KEY = "MEECAQAwEwYHKoZIzj0CAQYIKoZIzj0DAQcEJzAlAgEBBCCzKihManx3ughKcT5x8mdNj9GFGxH1UvKVKm8LqbDlig=="; + private static final String PUBLIC_KEY = "MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEqDGm5E0cL8w427d/3NujzqZPYvLR+dd6ZzyZaCmE3u9lN5lSjh7ia3xpxW0R20Gv6dxitwLBy02PhXsUk21B1Q=="; + + //SunEC secp384r1 + //private static final String PRIVATE_KEY = "ME4CAQAwEAYHKoZIzj0CAQYFK4EEACIENzA1AgEBBDAmegdlkADA/8rbG0CpuZdO1fnHOngWnBwOH04XFk0ZX/GNOTaYTtVMHuI/fuhALDk="; + //private static final String PUBLIC_KEY = "MHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEFFMlwova3OBpWS9YsZvjnQAuoLGwZjAGpNKFspZAskB2vTnkBOAmgm1I9UTM45rT5OzLPdrv9p+Ry76ZkEx7MK4s2eCj1U7RsLInZ1fFkgCleTDVYx/1JHBMf0wcscKR"; + + //SunEC secp521r1 + //private static final String PRIVATE_KEY = "MF8CAQAwEAYHKoZIzj0CAQYFK4EEACMESDBGAgEBBEHwZjYqD8oGXmtwvYtJI2mezVcU9hSKQHWc/LirGP/+oqVbnJQ8Unz6toGE8+WDXVFnIr6u4mmmV47L4/78Tmnu+w=="; + //private static final String PUBLIC_KEY = "MIGbMBAGByqGSM49AgEGBSuBBAAjA4GGAAQAUsovGsGwA3FFEfUOK01+UF1LFq2U7rIxzJYuW0/HVK18unQOkpFwezKzvwcYeS3hli2VHxHr6PJm2mVuh4MGtxYA4/2h46Dk5qMl5dX65723/4mq9sEHk9xQp0XtMfeeDyFGplNQYvtJ7OcqerOb3bhKcCFgoGGSFv8wwWjSI0NyiSU="; + + + /** Creates a new instance of ECDSAKeyLoader */ + + public SunECKeyLoader(int id, String configHome, boolean defaultKeys, String sigAlgorithm) { + this.id = id; + this.defaultKeys = defaultKeys; + this.sigAlgorithm = sigAlgorithm; + + if (configHome.equals("")) { + path = "config" + System.getProperty("file.separator") + "keysSunEC" + System.getProperty("file.separator"); + } else { + path = configHome + System.getProperty("file.separator") + "keysSunEC" + + System.getProperty("file.separator"); + } + + } + + /** + * Loads the public key of some processes from configuration files + * + * @return the PublicKey loaded from config/keys/publickey + * @throws Exception + * problems reading or parsing the key + */ + public PublicKey loadPublicKey(int id) + throws IOException, NoSuchAlgorithmException, InvalidKeySpecException, CertificateException { + + if (defaultKeys) { + //logger.debug("Loading default PublicKey, id: {}", id); + try { + logger.trace("Signature Algorithm: {}, Format: {} ", getPublicKeyFromString(PUBLIC_KEY).getAlgorithm(), + getPublicKeyFromString(PUBLIC_KEY).getFormat()); + } catch (NoSuchProviderException e1) { + // TODO Auto-generated catch block + e1.printStackTrace(); + } + try { + return getPublicKeyFromString(PUBLIC_KEY); + } catch (NoSuchProviderException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + } + + FileReader f = new FileReader(path + "publickey" + id); + BufferedReader r = new BufferedReader(f); + String tmp = ""; + String key = ""; + while ((tmp = r.readLine()) != null) { + key = key + tmp; + } + f.close(); + r.close(); + PublicKey ret = null; + logger.debug("Loading PublicKey from file, id: {}", id); + try { + ret = getPublicKeyFromString(key); + } catch (NoSuchProviderException e) { + e.printStackTrace(); + } + logger.trace("ID: {}, PublicKey Format: {}, PublicKey Algorithm: {} ", + new Object[] { id, ret.getFormat(), ret.getAlgorithm() }); + return ret; + } + + public PublicKey loadPublicKey() + throws IOException, NoSuchAlgorithmException, InvalidKeySpecException, CertificateException { + + if (defaultKeys) { + //logger.debug("Loading my default PublicKey, this.id: {}", this.id); + try { + logger.trace("Signature Algorithm: {}, Format: {} ", getPublicKeyFromString(PUBLIC_KEY).getAlgorithm(), + getPublicKeyFromString(PUBLIC_KEY).getFormat()); + } catch (NoSuchProviderException e1) { + e1.printStackTrace(); + } + + try { + return getPublicKeyFromString(PUBLIC_KEY); + } catch (NoSuchProviderException e) { + e.printStackTrace(); + } + } + logger.debug("Loading PublicKey from file, this.id: {}", this.id); + + FileReader f = new FileReader(path + "publickey" + this.id); + BufferedReader r = new BufferedReader(f); + String tmp = ""; + String key = ""; + while ((tmp = r.readLine()) != null) { + key = key + tmp; + } + f.close(); + r.close(); + PublicKey ret = null; + try { + ret = getPublicKeyFromString(PUBLIC_KEY); + } catch (NoSuchProviderException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + logger.trace("ID: {}, PublicKey Format: {}, PublicKey Algorithm: {} ", + new Object[] { this.id, ret.getFormat(), ret.getAlgorithm() }); + return ret; + + } + + /** + * Loads the private key of this process + * + * @return the PrivateKey loaded from config/keys/publickey + * @throws Exception + * problems reading or parsing the key + */ + public PrivateKey loadPrivateKey() throws IOException, NoSuchAlgorithmException, InvalidKeySpecException { + + if (defaultKeys) { + //logger.debug("Loading default PrivateKey, ID: {}", this.id); + try { + return getPrivateKeyFromString(PRIVATE_KEY); + } catch (NoSuchProviderException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + } + + if (privateKey == null) { + FileReader f = new FileReader(path + "privatekey" + this.id); + BufferedReader r = new BufferedReader(f); + String tmp = ""; + String key = ""; + while ((tmp = r.readLine()) != null) { + key = key + tmp; + } + f.close(); + r.close(); + logger.debug("Loading first time PrivateKey from file, this.id: {}", this.id); + try { + privateKey = getPrivateKeyFromString(key); + logger.trace("PrivateKey loaded for this.id: {}, " + + "PrivateKey Format: {}, " + + "PrivateKey Algorithm: {} ", + new Object[] { this.id, privateKey.getFormat(), privateKey.getAlgorithm()}); + } catch (NoSuchProviderException e) { + e.printStackTrace(); + } + } + logger.trace("Returning previous stored PrivateKey from file, this.id: {}", this.id); + return privateKey; + } + + // utility methods for going from string to public/private key + private PrivateKey getPrivateKeyFromString(String key) + throws NoSuchProviderException, NoSuchAlgorithmException, InvalidKeySpecException { + + KeyFactory keyFactory = KeyFactory.getInstance("EC", "SunEC"); + EncodedKeySpec privateKeySpec = new PKCS8EncodedKeySpec(Base64.decodeBase64(key)); + privateKey = keyFactory.generatePrivate(privateKeySpec); + return privateKey; + } + + private PublicKey getPublicKeyFromString(String key) + throws NoSuchProviderException, NoSuchAlgorithmException, InvalidKeySpecException, NoSuchProviderException { + KeyFactory keyFactory = KeyFactory.getInstance("EC", "SunEC"); + X509EncodedKeySpec pubKeySpec = new X509EncodedKeySpec(Base64.decodeBase64(key)); + PublicKey publicKey = keyFactory.generatePublic(pubKeySpec); + return publicKey; + } + + @Override + public String getSignatureAlgorithm() { + return this.sigAlgorithm; + } + +} diff --git a/src/bftsmart/reconfiguration/util/TOMConfiguration.java b/src/bftsmart/reconfiguration/util/TOMConfiguration.java index 2219a4c94..e13237913 100644 --- a/src/bftsmart/reconfiguration/util/TOMConfiguration.java +++ b/src/bftsmart/reconfiguration/util/TOMConfiguration.java @@ -16,7 +16,6 @@ package bftsmart.reconfiguration.util; import bftsmart.tom.util.KeyLoader; -import java.security.Provider; import java.util.StringTokenizer; import java.util.regex.Pattern; @@ -31,6 +30,7 @@ public class TOMConfiguration extends Configuration { protected int n; protected int f; protected int requestTimeout; + protected int batchTimeout; protected int tomPeriod; protected int paxosHighMark; protected int revivalHighMark; @@ -43,8 +43,7 @@ public class TOMConfiguration extends Configuration { protected boolean shutdownHookEnabled; protected boolean useSenderThread; private int numNIOThreads; - private int useMACs; - private int useSignatures; + private boolean useSignatures; private boolean stateTransferEnabled; private int checkpointPeriod; private int globalCheckpointPeriod; @@ -63,14 +62,21 @@ public class TOMConfiguration extends Configuration { private boolean sameBatchSize; private String bindAddress; + /* Tulio Ribeiro*/ + //private Boolean ssltls=true; + private String ssltlsProtocolVersion; + private String keyStoreFile; + private String [] enabledCiphers; + + /** Creates a new instance of TOMConfiguration */ - public TOMConfiguration(int processId, KeyLoader loader, Provider provider) { - super(processId, loader, provider); + public TOMConfiguration(int processId, KeyLoader loader) { + super(processId, loader); } /** Creates a new instance of TOMConfiguration */ - public TOMConfiguration(int processId, String configHome, KeyLoader loader, Provider provider) { - super(processId, configHome, loader, provider); + public TOMConfiguration(int processId, String configHome, KeyLoader loader) { + super(processId, configHome, loader); } @@ -105,6 +111,16 @@ protected void init() { requestTimeout = 0; } } + + s = (String) configs.remove("system.totalordermulticast.batchtimeout"); + if (s == null) { + batchTimeout = -1; + } else { + batchTimeout = Integer.parseInt(s); + if (batchTimeout <= 0) { + batchTimeout = -1; + } + } s = (String) configs.remove("system.totalordermulticast.highMark"); if (s == null) { @@ -171,18 +187,11 @@ protected void init() { numNIOThreads = Integer.parseInt(s); } - s = (String) configs.remove("system.communication.useMACs"); - if (s == null) { - useMACs = 0; - } else { - useMACs = Integer.parseInt(s); - } - s = (String) configs.remove("system.communication.useSignatures"); if (s == null) { - useSignatures = 0; + useSignatures = false; } else { - useSignatures = Integer.parseInt(s); + useSignatures = Boolean.parseBoolean(s); } s = (String) configs.remove("system.totalordermulticast.state_transfer"); @@ -337,6 +346,55 @@ protected void init() { sameBatchSize = false; } + /** + * Tulio Ribeiro + * + * SSL/TLS configuration parameters. + * Default values: + * # keyStoreFile = "EC_KeyPair_256.pkcs12"; + * # enabledCiphers = new String[] {"TLS_RSA_WITH_NULL_SHA256", "TLS_ECDHE_ECDSA_WITH_NULL_SHA"}; + * # ssltlsProtocolVersion = "TLSv1.2"; + */ + + + s = (String) configs.remove("system.ssltls.key_store_file"); + if(s == null){ + keyStoreFile = "EC_KeyPair_256.pkcs12"; + }else{ + keyStoreFile = s; + } + + s = (String) configs.remove("system.ssltls.enabled_ciphers"); + if(s == null){ + enabledCiphers = new String[] {"TLS_RSA_WITH_NULL_SHA256", "TLS_ECDHE_ECDSA_WITH_NULL_SHA"}; + }else{ + enabledCiphers = s.split(","); + } + + s = (String) configs.remove("system.ssltls.protocol_version"); + if (s == null) { + ssltlsProtocolVersion = "TLSv1.2"; + } else { + switch (s) { + case "SSLv3": + ssltlsProtocolVersion = "SSLv3"; + break; + case "TLSv1": + ssltlsProtocolVersion = "TLSv1"; + break; + case "TLSv1.1": + ssltlsProtocolVersion = "TLSv1.1"; + break; + case "TLSv1.2": + ssltlsProtocolVersion = "TLSv1.2"; + break; + default: + System.out.println("Setting default SSL/TLS protocol version: TLSv1.2"); + ssltlsProtocolVersion = "TLSv1.2"; + break; + } + } + } catch (Exception e) { logger.error("Could not parse system configuration file",e); } @@ -369,6 +427,10 @@ public int getRequestTimeout() { return requestTimeout; } + public int getBatchTimeout() { + return batchTimeout; + } + public int getReplyVerificationTime() { return replyVerificationTime; } @@ -430,19 +492,12 @@ public int getNumberOfNonces() { } /** - * Indicates if signatures should be used (1) or not (0) to authenticate client requests + * Indicates if signatures should be used true or not false to authenticate client requests */ - public int getUseSignatures() { + public boolean getUseSignatures() { return useSignatures; } - /** - * Indicates if MACs should be used (1) or not (0) to authenticate client-server and server-server messages - */ - public int getUseMACs() { - return useMACs; - } - /** * Indicates the checkpoint period used when fetching the state from the application */ @@ -509,4 +564,20 @@ public boolean getSameBatchSize() { public String getBindAddress() { return bindAddress; } + + /** + * Tulio Ribeiro ## SSL/TLS getters. + * */ + public String getSSLTLSProtocolVersion() { + return ssltlsProtocolVersion; + } + + public String getSSLTLSKeyStore() { + return keyStoreFile; + } + + public String[] getEnabledCiphers() { + return enabledCiphers; + } + } diff --git a/src/bftsmart/reconfiguration/views/DefaultViewStorage.java b/src/bftsmart/reconfiguration/views/DefaultViewStorage.java index b15ff5ab8..25f030df5 100644 --- a/src/bftsmart/reconfiguration/views/DefaultViewStorage.java +++ b/src/bftsmart/reconfiguration/views/DefaultViewStorage.java @@ -30,15 +30,15 @@ public class DefaultViewStorage implements ViewStorage { private String path = ""; - - public DefaultViewStorage() { - String sep = System.getProperty("file.separator"); - path = System.getProperty("user.dir") + sep + "config"; + + public DefaultViewStorage(String configPath) { + + path = configPath; File f = new File(path); if (!f.exists()) { f.mkdirs(); } - path = path + sep + "currentView"; + path = path + System.getProperty("file.separator") + "currentView"; } @Override diff --git a/src/bftsmart/reconfiguration/views/TestViewSerialization.java b/src/bftsmart/reconfiguration/views/TestViewSerialization.java index f57008791..bf97904cb 100644 --- a/src/bftsmart/reconfiguration/views/TestViewSerialization.java +++ b/src/bftsmart/reconfiguration/views/TestViewSerialization.java @@ -30,7 +30,10 @@ public static void main(String[] args) throws Exception { in[2] = new InetSocketAddress("127.0.0.1",1234); in[3] = new InetSocketAddress("127.0.0.1",1234); View v = new View(10, ids,1,in); - ViewStorage st = new DefaultViewStorage(); + + String path = System.getProperty("user.dir") + System.getProperty("file.separator") + "config"; + + ViewStorage st = new DefaultViewStorage(path); st.storeView(v); View r = st.readView(); diff --git a/src/bftsmart/statemanagement/StateManager.java b/src/bftsmart/statemanagement/StateManager.java index e6c85617c..725b9ca12 100644 --- a/src/bftsmart/statemanagement/StateManager.java +++ b/src/bftsmart/statemanagement/StateManager.java @@ -1,53 +1,383 @@ /** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ + * Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and + * the authors indicated in the @author tags + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. You may obtain a copy of + * the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations under + * the License. + */ package bftsmart.statemanagement; +import java.util.Collection; +import java.util.HashMap; +import java.util.Arrays; +import java.util.Set; + +import bftsmart.consensus.messages.ConsensusMessage; +import bftsmart.reconfiguration.ServerViewController; +import bftsmart.reconfiguration.views.View; +import bftsmart.statemanagement.standard.StandardSMMessage; import bftsmart.tom.core.DeliveryThread; +import bftsmart.tom.core.ExecutionManager; import bftsmart.tom.core.TOMLayer; +import bftsmart.tom.leaderchange.LCManager; +import bftsmart.tom.leaderchange.CertifiedDecision; +import bftsmart.tom.util.TOMUtil; +import bftsmart.tom.server.defaultservices.DefaultApplicationState; +import java.util.Map; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** - * TODO: Don't know if this class will be used. For now, leave it here * - * Check if the changes for supporting dynamicity are correct - * - * @author Joao Sousa + * @author Marcel Santos + * */ -public interface StateManager { - - public void requestAppState(int cid); +public abstract class StateManager { - public void analyzeState(int cid); + private Logger logger = LoggerFactory.getLogger(this.getClass()); - public void stateTimeout(); - - public void init(TOMLayer tomLayer, DeliveryThread dt); - - public void SMRequestDeliver(SMMessage msg, boolean isBFT); - - public void SMReplyDeliver(SMMessage msg, boolean isBFT); + protected TOMLayer tomLayer; + protected ServerViewController SVController; + protected DeliveryThread dt; + protected ExecutionManager execManager; - public void askCurrentConsensusId(); - - public void currentConsensusIdAsked(int sender); + + protected HashMap senderStates = null; + protected HashMap senderViews = null; + protected HashMap senderRegencies = null; + protected HashMap senderLeaders = null; + protected HashMap senderProofs = null; + + protected boolean appStateOnly; + protected int waitingCID = -1; + protected int queryID = -1; + protected int lastCID; + protected ApplicationState state; + + protected boolean isInitializing = true; + protected Map> queries = new HashMap<>(); + + public StateManager() { + senderStates = new HashMap<>(); + senderViews = new HashMap<>(); + senderRegencies = new HashMap<>(); + senderLeaders = new HashMap<>(); + senderProofs = new HashMap<>(); + } + + protected int getReplies() { + return senderStates.size(); + } + + protected boolean enoughReplies() { + return senderStates.size() > SVController.getCurrentViewF(); + } + + protected boolean enoughRegencies(int regency) { + + Collection regencies = senderRegencies.values(); + int counter = 0; + for (int r : regencies) { + if (regency == r) { + counter++; + } + } + boolean result = counter > SVController.getQuorum(); + return result; + } + + protected boolean enoughLeaders(int leader) { + + Collection leaders = senderLeaders.values(); + int counter = 0; + for (int l : leaders) { + if (leader == l) { + counter++; + } + } + boolean result = counter > SVController.getQuorum(); + return result; + + } + + protected boolean enoughViews(View view) { + Collection views = senderViews.values(); + int counter = 0; + for (View v : views) { + if (view.equals(v)) { + counter++; + } + } + boolean result = counter > SVController.getQuorum(); + return result; + } - public void currentConsensusIdReceived(SMMessage msg); + // check if the consensus messages are consistent without checking the mac/signatures + // if it is consistent, it returns the respective consensus ID; otherwise, returns -1 + private int proofIsConsistent(Set proof) { + + int id = -1; + byte[] value = null; + + for (ConsensusMessage cm : proof) { + + if (id == -1) id = cm.getNumber(); + if (value == null) value = cm.getValue(); + + if (id != cm.getNumber() || !Arrays.equals(value, cm.getValue())) { + return -1; // they are not consistent, so the proof is invalid + } + + } + + // if the values are still these, this means the proof is empty, thus is invalid + if (id == -1 || value == null) return -1; + + return id; + } + + protected boolean enoughProofs(int cid, LCManager lc) { + + int counter = 0; + for (CertifiedDecision cDec : senderProofs.values()) { + + if (cDec != null && cid == proofIsConsistent(cDec.getConsMessages()) && lc.hasValidProof(cDec)) { + counter++; + } + + } + boolean result = counter > SVController.getQuorum(); + return result; + } - public void setLastCID(int lastCID); + /** + * Clear the collections and state hold by this object. Calls clear() in the + * States, Leaders, regencies and Views collections. Sets the state to + * null; + */ + protected void reset() { + senderStates.clear(); + senderLeaders.clear(); + senderRegencies.clear(); + senderViews.clear(); + senderProofs.clear(); + state = null; + } + + public Collection receivedStates() { + return senderStates.values(); + } + + public void setLastCID(int cid) { + lastCID = cid; + } + + public int getLastCID() { + return lastCID; + } + + public void requestAppState(int cid) { + lastCID = cid + 1; + waitingCID = cid; + logger.debug("Updated waitingcid to " + cid); + appStateOnly = true; + requestState(); + } + + public void analyzeState(int cid) { + logger.debug("The state transfer protocol is enabled"); + if (waitingCID == -1) { + logger.debug("I'm not waiting for any state, so I will keep record of this message"); + if (tomLayer.execManager.isDecidable(cid)) { + logger.info("I have now more than " + SVController.getCurrentViewF() + " messages for CID " + cid + " which are beyond CID " + lastCID); + lastCID = cid; + waitingCID = cid - 1; + logger.info("I will be waiting for state messages associated to consensus " + waitingCID); + requestState(); + } + } + } + + public boolean isRetrievingState() { + if (isInitializing) { + return true; + } + return waitingCID > -1; + } + + public void askCurrentConsensusId() { + + if (SVController.getCurrentViewN() == 1) { + logger.info("Replica state is up to date"); + dt.deliverLock(); + isInitializing = false; + tomLayer.setLastExec(-1); + dt.canDeliver(); + dt.deliverUnlock(); + return; + } + + int me = SVController.getStaticConf().getProcessId(); + int[] target = SVController.getCurrentViewOtherAcceptors(); + SMMessage currentCID; + + while (isInitializing) { + + logger.debug("Sending CID query with ID {} to replicas {}", queryID, target); + + queryID++; + + currentCID = new StandardSMMessage(me, queryID, TOMUtil.SM_ASK_INITIAL, 0, null, null, 0, 0); + + tomLayer.getCommunication().send(target, currentCID); + try { + Thread.sleep(1000); + } catch (InterruptedException e) { + logger.error("Interruption during sleep",e); + } + } + } + + public void currentConsensusIdAsked(int sender, int id) { + + logger.debug("Received CID query from {} with ID {}",sender,id); + + int me = SVController.getStaticConf().getProcessId(); + int lastConsensusId = tomLayer.getLastExec(); + + DefaultApplicationState state = new DefaultApplicationState(null, -1, lastConsensusId, null, null, -1); + + SMMessage currentCIDReply = new StandardSMMessage(me, id, TOMUtil.SM_REPLY_INITIAL, 0, state, null, 0, 0); + tomLayer.getCommunication().send(new int[]{sender}, currentCIDReply); + + logger.debug("Sent CID reply to replica {} with ID {}",sender,id); + } + + public synchronized void currentConsensusIdReceived(SMMessage smsg) { + + logger.debug("Received CID reply from replica {} with ID {} (expecting ID {})",smsg.getSender(), smsg.getCID(), queryID); + + if (!isInitializing || waitingCID > -1 || queryID != smsg.getCID()) { + + logger.debug("Ignoring CID query from {} with ID {}",smsg.getSender(), smsg.getCID()); + + return; + } + + Map replies = queries.get(queryID); + + if (replies == null) { + + replies = new HashMap<>(); + queries.put(queryID, replies); + } + + replies.put(smsg.getSender(), smsg.getState().getLastCID()); + + logger.debug("Received {} replies for query ID {}",replies.size(),queryID); + + if (replies.size() > SVController.getQuorum()) { + + logger.debug("Received quorum of replies for query ID {}", queryID); + + HashMap cids = new HashMap<>(); + for (int id : replies.keySet()) { + + int value = replies.get(id); + + Integer count = cids.get(value); + if (count == null) { + cids.put(value, 1); + } else { + cids.put(value, count + 1); + } + } + for (int cid : cids.keySet()) { + + logger.debug("CID {} came from {} replicas", cid, cids.get(cid)); + + if (cids.get(cid) > SVController.getQuorum()) { + + logger.debug("There is a quorum for CID {}",cid); + + queries.clear(); + + if (cid == lastCID) { + logger.info("Replica state is up to date"); + dt.deliverLock(); + isInitializing = false; + tomLayer.setLastExec(cid); + dt.canDeliver(); + dt.deliverUnlock(); + break; + } else { + //ask for state + logger.info("Requesting state from other replicas"); + lastCID = cid + 1; + if (waitingCID == -1) { + waitingCID = cid; + requestState(); + } + } + } + } + } + } - public int getLastCID(); + public void init(TOMLayer tomLayer, DeliveryThread dt) { + SVController = tomLayer.controller; + + this.tomLayer = tomLayer; + this.dt = dt; + this.execManager = tomLayer.execManager; + + state = null; + lastCID = -1; + waitingCID = -1; + + appStateOnly = false; + } - public boolean isRetrievingState(); + public void triggerTimeout(SMMessage msg) { + + int[] myself = new int[1]; + myself[0] = SVController.getStaticConf().getProcessId(); + tomLayer.getCommunication().send(myself, msg); + } + + /** + * Request the state to the other replicas. It should ask for the state defined in the 'waitingCID' variable. + */ + protected abstract void requestState(); + + /** + * To use if the state manager needs to use timeout for liveness and when such timeout expires. To trigger the invocation, + * the method 'triggerTimeout' should be invoked by the class extending StateManager supplying an SMMessage of type 'TRIGGER_SM_LOCALLY' + */ + public abstract void stateTimeout(); + + /** + * Invoked when a replica is asking to be sent the application state. + * @param msg The message sent by the replica, of type 'SM_REQUEST'. + * @param isBFT true if the library is set for BFT, false if CFT + */ + public abstract void SMRequestDeliver(SMMessage msg, boolean isBFT); + + /** + * Invoked when a replica receives a reply to its request to be sent the application state. + * @param msg The message sent by the replica, of type 'SM_REPLY'. + * @param isBFT true if the library is set for BFT, false if CFT + */ + public abstract void SMReplyDeliver(SMMessage msg, boolean isBFT); + } diff --git a/src/bftsmart/statemanagement/strategy/durability/CSTRequest.java b/src/bftsmart/statemanagement/durability/CSTRequest.java similarity index 96% rename from src/bftsmart/statemanagement/strategy/durability/CSTRequest.java rename to src/bftsmart/statemanagement/durability/CSTRequest.java index c10805f51..b3c499ff8 100644 --- a/src/bftsmart/statemanagement/strategy/durability/CSTRequest.java +++ b/src/bftsmart/statemanagement/durability/CSTRequest.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.statemanagement.strategy.durability; +package bftsmart.statemanagement.durability; import java.io.Serializable; diff --git a/src/bftsmart/statemanagement/strategy/durability/CSTRequestF1.java b/src/bftsmart/statemanagement/durability/CSTRequestF1.java similarity index 98% rename from src/bftsmart/statemanagement/strategy/durability/CSTRequestF1.java rename to src/bftsmart/statemanagement/durability/CSTRequestF1.java index be9ba46c6..06c8e51ed 100644 --- a/src/bftsmart/statemanagement/strategy/durability/CSTRequestF1.java +++ b/src/bftsmart/statemanagement/durability/CSTRequestF1.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.statemanagement.strategy.durability; +package bftsmart.statemanagement.durability; import java.net.InetSocketAddress; diff --git a/src/bftsmart/statemanagement/strategy/durability/CSTRequestFGT1.java b/src/bftsmart/statemanagement/durability/CSTRequestFGT1.java similarity index 98% rename from src/bftsmart/statemanagement/strategy/durability/CSTRequestFGT1.java rename to src/bftsmart/statemanagement/durability/CSTRequestFGT1.java index cd6dfeca4..63b33eefa 100644 --- a/src/bftsmart/statemanagement/strategy/durability/CSTRequestFGT1.java +++ b/src/bftsmart/statemanagement/durability/CSTRequestFGT1.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.statemanagement.strategy.durability; +package bftsmart.statemanagement.durability; /** * This class is used to define the roles in the Collaborative State Transfer protocol. diff --git a/src/bftsmart/statemanagement/strategy/durability/CSTSMMessage.java b/src/bftsmart/statemanagement/durability/CSTSMMessage.java similarity index 96% rename from src/bftsmart/statemanagement/strategy/durability/CSTSMMessage.java rename to src/bftsmart/statemanagement/durability/CSTSMMessage.java index 304afe3f3..033b8199d 100644 --- a/src/bftsmart/statemanagement/strategy/durability/CSTSMMessage.java +++ b/src/bftsmart/statemanagement/durability/CSTSMMessage.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.statemanagement.strategy.durability; +package bftsmart.statemanagement.durability; import java.io.IOException; import java.io.ObjectInput; diff --git a/src/bftsmart/statemanagement/strategy/durability/CSTState.java b/src/bftsmart/statemanagement/durability/CSTState.java similarity index 98% rename from src/bftsmart/statemanagement/strategy/durability/CSTState.java rename to src/bftsmart/statemanagement/durability/CSTState.java index 23473cfcd..0ef4f0fa4 100644 --- a/src/bftsmart/statemanagement/strategy/durability/CSTState.java +++ b/src/bftsmart/statemanagement/durability/CSTState.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.statemanagement.strategy.durability; +package bftsmart.statemanagement.durability; import bftsmart.consensus.messages.ConsensusMessage; import bftsmart.reconfiguration.ServerViewController; @@ -122,7 +122,7 @@ public CertifiedDecision getCertifiedDecision(ServerViewController controller) { //Serialize the TOMMessages to re-create the proposed value BatchBuilder bb = new BatchBuilder(0); byte[] value = bb.makeBatch(requests, ci.msgCtx[0].getNumOfNonces(), - ci.msgCtx[0].getSeed(), ci.msgCtx[0].getTimestamp(), controller); + ci.msgCtx[0].getSeed(), ci.msgCtx[0].getTimestamp(), controller.getStaticConf().getUseSignatures()); //Assemble and return the certified decision return new CertifiedDecision(pid, getLastCID(), value, proof); diff --git a/src/bftsmart/statemanagement/durability/DurableStateManager.java b/src/bftsmart/statemanagement/durability/DurableStateManager.java new file mode 100644 index 000000000..4fb1aba45 --- /dev/null +++ b/src/bftsmart/statemanagement/durability/DurableStateManager.java @@ -0,0 +1,441 @@ +/** + * Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package bftsmart.statemanagement.durability; + +import java.io.IOException; +import java.io.ObjectInputStream; +import java.net.InetSocketAddress; +import java.net.Socket; +import java.net.UnknownHostException; +import java.util.Arrays; +import java.util.Queue; +import java.util.Timer; +import java.util.TimerTask; +import java.util.concurrent.locks.ReentrantLock; + +import bftsmart.consensus.messages.ConsensusMessage; +import bftsmart.reconfiguration.views.View; +import bftsmart.statemanagement.ApplicationState; +import bftsmart.statemanagement.SMMessage; +import bftsmart.statemanagement.StateManager; +import bftsmart.tom.server.defaultservices.CommandsInfo; +import bftsmart.tom.server.durability.DurabilityCoordinator; +import bftsmart.tom.util.TOMUtil; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class DurableStateManager extends StateManager { + + private Logger logger = LoggerFactory.getLogger(this.getClass()); + + private ReentrantLock lockTimer = new ReentrantLock(); + private Timer stateTimer = null; + private final static long INIT_TIMEOUT = 40000; + private long timeout = INIT_TIMEOUT; + + private CSTRequestF1 cstRequest; + + private CSTState stateCkp; + private CSTState stateLower; + private CSTState stateUpper; + + private Thread stateThread = null; + + public void setLastCID(int cid) { + + super.setLastCID(cid); + tomLayer.setLastExec(cid); + } + + @Override + protected void requestState() { + if (tomLayer.requestsTimer != null) { + tomLayer.requestsTimer.clearAll(); + } + + int myProcessId = SVController.getStaticConf().getProcessId(); + int[] otherProcesses = SVController.getCurrentViewOtherAcceptors(); + int globalCkpPeriod = SVController.getStaticConf() + .getGlobalCheckpointPeriod(); + + CSTRequestF1 cst = new CSTRequestF1(waitingCID); + cst.defineReplicas(otherProcesses, globalCkpPeriod, myProcessId); + this.cstRequest = cst; + CSTSMMessage cstMsg = new CSTSMMessage(myProcessId, waitingCID, + TOMUtil.SM_REQUEST, cst, null, null, -1, -1); + tomLayer.getCommunication().send( + SVController.getCurrentViewOtherAcceptors(), cstMsg); + + logger.info("I just sent a request to the other replicas for the state up to CID " + + waitingCID); + + TimerTask stateTask = new TimerTask() { + public void run() { + + logger.info("Timeout to retrieve state"); + + CSTSMMessage msg = new CSTSMMessage(-1, waitingCID,TOMUtil.TRIGGER_SM_LOCALLY, null, null, null, -1, -1); + + triggerTimeout(msg); + } + }; + + stateTimer = new Timer("state timer"); + timeout = timeout * 2; + stateTimer.schedule(stateTask, timeout); + } + + @Override + public void stateTimeout() { + lockTimer.lock(); + logger.debug("(StateManager.stateTimeout) Timeout for the replica that was supposed to send the complete state. Changing desired replica."); + if (stateTimer != null) { + stateTimer.cancel(); + } + reset(); + requestState(); + lockTimer.unlock(); + } + + @Override + public void SMRequestDeliver(SMMessage msg, boolean isBFT) { + logger.debug("Invoked method"); + if (SVController.getStaticConf().isStateTransferEnabled() + && dt.getRecoverer() != null) { + logger.debug("The state transfer protocol is enabled"); + logger.debug("I received a state request for CID " + + msg.getCID() + " from replica " + msg.getSender()); + CSTSMMessage cstMsg = (CSTSMMessage) msg; + CSTRequestF1 cstConfig = cstMsg.getCstConfig(); + boolean sendState = cstConfig.getCheckpointReplica() == SVController + .getStaticConf().getProcessId(); + if (sendState) { + logger.debug("I should be the one sending the state"); + } + + logger.info("State asked by replica " + msg.getSender()); + + int[] targets = {msg.getSender()}; + InetSocketAddress address = SVController.getCurrentView().getAddress( + SVController.getStaticConf().getProcessId()); + String myIp = address.getHostName(); + int myId = SVController.getStaticConf().getProcessId(); + int port = 4444 + myId; + address = new InetSocketAddress(myIp, port); + cstConfig.setAddress(address); + CSTSMMessage reply = new CSTSMMessage(myId, msg.getCID(), + TOMUtil.SM_REPLY, cstConfig, null, + SVController.getCurrentView(), tomLayer.getSynchronizer().getLCManager().getLastReg(), + tomLayer.execManager.getCurrentLeader()); + + tomLayer.getCommunication().send(targets, reply); + + if (stateThread == null) { + + StateSenderServer stateServer = new StateSenderServer(port); + stateServer.setRecoverable(dt.getRecoverer()); + stateServer.setRequest(cstConfig); + stateThread = new Thread(stateServer); + stateThread.start(); + } + } + } + + @Override + public void SMReplyDeliver(SMMessage msg, boolean isBFT) { + lockTimer.lock(); + CSTSMMessage reply = (CSTSMMessage) msg; + if (SVController.getStaticConf().isStateTransferEnabled()) { + logger.debug("The state transfer protocol is enabled"); + logger.info("I received a state reply for CID " + + reply.getCID() + + " from replica " + + reply.getSender()); + + logger.info("Received CID: " + reply.getCID() + + ". Waiting " + waitingCID); + if (waitingCID != -1 && reply.getCID() == waitingCID) { + + int currentRegency = -1; + int currentLeader = -1; + View currentView = null; + // CertifiedDecision currentProof = null; + + if (!appStateOnly) { + senderRegencies.put(reply.getSender(), reply.getRegency()); + senderLeaders.put(reply.getSender(), reply.getLeader()); + senderViews.put(reply.getSender(), reply.getView()); + // senderProofs.put(msg.getSender(), msg.getState().getCertifiedDecision(SVController)); + if (enoughRegencies(reply.getRegency())) { + currentRegency = reply.getRegency(); + } + if (enoughLeaders(reply.getLeader())) { + currentLeader = reply.getLeader(); + } + if (enoughViews(reply.getView())) { + currentView = reply.getView(); + if (!currentView.isMember(SVController.getStaticConf() + .getProcessId())) { + logger.warn("Not a member!"); + } + } + // if (enoughProofs(waitingCID, this.tomLayer.getSynchronizer().getLCManager())) currentProof = msg.getState().getCertifiedDecision(SVController); + + } else { + currentLeader = tomLayer.execManager.getCurrentLeader(); + currentRegency = tomLayer.getSynchronizer().getLCManager().getLastReg(); + currentView = SVController.getCurrentView(); + } + + logger.debug("The reply is for the CID that I want!"); + + InetSocketAddress address = reply.getCstConfig().getAddress(); + Socket clientSocket; + ApplicationState stateReceived = null; + try { + clientSocket = new Socket(address.getHostName(), + address.getPort()); + ObjectInputStream in = new ObjectInputStream( + clientSocket.getInputStream()); + stateReceived = (ApplicationState) in.readObject(); + } catch (UnknownHostException e) { + // TODO Auto-generated catch block + logger.error("Failed to connect to address", e); + } catch (IOException e) { + // TODO Auto-generated catch block + logger.error("Failed to connect to address", e); + } catch (ClassNotFoundException e) { + // TODO Auto-generated catch block + logger.error("Failed to deserialize application state object", e); + } + + if (stateReceived instanceof CSTState) { + senderStates.put(reply.getSender(), stateReceived); + if (reply.getSender() == cstRequest.getCheckpointReplica()) { + this.stateCkp = (CSTState) stateReceived; + } + if (reply.getSender() == cstRequest.getLogLower()) { + this.stateLower = (CSTState) stateReceived; + } + if (reply.getSender() == cstRequest.getLogUpper()) { + this.stateUpper = (CSTState) stateReceived; + } + } + + if (senderStates.size() == 3) { + + CommandsInfo[] lowerLog = stateLower.getLogLower(); + CommandsInfo[] upperLog = stateUpper.getLogUpper(); + logger.info("lowerLog "); + if (lowerLog != null) { + logger.info("\t" + lowerLog.length); + } + logger.info("upperLog "); + if (upperLog != null) { + logger.info("\t" + upperLog.length); + } + + boolean haveState = false; + byte[] lowerbytes = TOMUtil.getBytes(lowerLog); + logger.debug("Log lower bytes size: " + + lowerbytes.length); + byte[] upperbytes = TOMUtil.getBytes(upperLog); + logger.debug("Log upper bytes size: " + + upperbytes.length); + + byte[] lowerLogHash = TOMUtil.computeHash(lowerbytes); + byte[] upperLogHash = TOMUtil.computeHash(upperbytes); + + // validate lower log + if (Arrays.equals(stateCkp.getHashLogLower(), lowerLogHash)) { + haveState = true; + } else { + logger.error("Lower log does not match"); + } + // validate upper log + if (!haveState || !Arrays.equals(stateCkp.getHashLogUpper(), upperLogHash)) { + haveState = false; + logger.error("Upper log does not match"); + } + + CSTState statePlusLower = new CSTState(stateCkp.getSerializedState(), + TOMUtil.getBytes(stateCkp.getSerializedState()), + stateLower.getLogLower(), stateCkp.getHashLogLower(), null, null, + stateCkp.getCheckpointCID(), stateUpper.getCheckpointCID(), SVController.getStaticConf().getProcessId()); + + if (haveState) { // validate checkpoint + logger.info("validating checkpoint!!!"); + dt.getRecoverer().setState(statePlusLower); + byte[] currentStateHash = ((DurabilityCoordinator) dt.getRecoverer()).getCurrentStateHash(); + if (!Arrays.equals(currentStateHash, stateUpper.getHashCheckpoint())) { + logger.error("ckp hash don't match"); + haveState = false; + } + } + + logger.info("Current regency: " + currentRegency); + logger.info("Current leader: " + currentLeader); + logger.info("Current view: " + currentView); + if (currentRegency > -1 && currentLeader > -1 + && currentView != null && haveState && (!isBFT || /*currentProof != null ||*/ appStateOnly)) { + logger.info("---- RECEIVED VALID STATE ----"); + + logger.debug("The state of those replies is good!"); + logger.debug("CID State requested: " + reply.getCID()); + logger.debug("CID State received: " + stateUpper.getLastCID()); + + tomLayer.getSynchronizer().getLCManager().setLastReg(currentRegency); + tomLayer.getSynchronizer().getLCManager().setNextReg(currentRegency); + tomLayer.getSynchronizer().getLCManager().setNewLeader(currentLeader); + + tomLayer.execManager.setNewLeader(currentLeader); + +// if (currentProof != null && !appStateOnly) { +// +// System.out.println("Installing proof for consensus " + waitingCID); +// +// Consensus cons = execManager.getConsensus(waitingCID); +// Epoch e = null; +// +// for (ConsensusMessage cm : currentProof.getConsMessages()) { +// +// e = cons.getEpoch(cm.getEpoch(), true, SVController); +// if (e.getTimestamp() != cm.getEpoch()) { +// +// System.out.println("Strange... proof contains messages from more than just one epoch"); +// e = cons.getEpoch(cm.getEpoch(), true, SVController); +// } +// e.addToProof(cm); +// +// if (cm.getType() == MessageFactory.ACCEPT) { +// e.setAccept(cm.getSender(), cm.getValue()); +// } +// +// else if (cm.getType() == MessageFactory.WRITE) { +// e.setWrite(cm.getSender(), cm.getValue()); +// } +// +// } +// +// +// if (e != null) { +// +// byte[] hash = tomLayer.computeHash(currentProof.getDecision()); +// e.propValueHash = hash; +// e.propValue = currentProof.getDecision(); +// e.deserializedPropValue = tomLayer.checkProposedValue(currentProof.getDecision(), false); +// cons.decided(e, false); +// +// System.out.println("Successfully installed proof for consensus " + waitingCID); +// +// } else { +// System.out.println("Failed to install proof for consensus " + waitingCID); +// +// } +// +// } + // I might have timed out before invoking the state transfer, so + // stop my re-transmission of STOP messages for all regencies up to the current one + if (currentRegency > 0) { + tomLayer.getSynchronizer().removeSTOPretransmissions(currentRegency - 1); + } + + logger.debug("trying to acquire deliverlock"); + dt.deliverLock(); + logger.debug("acquired"); + + // this makes the isRetrievingState() evaluates to false + waitingCID = -1; + dt.update(stateUpper); + + // Deal with stopped messages that may come from + // synchronization phase + if (!appStateOnly && execManager.stopped()) { + Queue stoppedMsgs = execManager.getStoppedMsgs(); + for (ConsensusMessage stopped : stoppedMsgs) { + if (stopped.getNumber() > state.getLastCID()) { + execManager.addOutOfContextMessage(stopped); + } + } + execManager.clearStopped(); + execManager.restart(); + } + + logger.info("Processing out of context messages"); + tomLayer.processOutOfContext(); + + if (SVController.getCurrentViewId() != currentView.getId()) { + logger.info("Installing current view!"); + SVController.reconfigureTo(currentView); + } + + isInitializing = false; + + dt.canDeliver(); + dt.deliverUnlock(); + + reset(); + + logger.info("I updated the state!"); + + tomLayer.requestsTimer.Enabled(true); + tomLayer.requestsTimer.startTimer(); + if (stateTimer != null) { + stateTimer.cancel(); + } + + if (appStateOnly) { + appStateOnly = false; + tomLayer.getSynchronizer().resumeLC(); + } + } else if (state == null + && (SVController.getCurrentViewN() / 2) < getReplies()) { + logger.warn("---- DIDNT RECEIVE STATE ----"); + + logger.debug("I have more than " + + (SVController.getCurrentViewN() / 2) + + " messages that are no good!"); + + waitingCID = -1; + reset(); + + if (stateTimer != null) { + stateTimer.cancel(); + } + + if (appStateOnly) { + requestState(); + } + } else if (!haveState) { + logger.warn("---- RECEIVED INVALID STATE ----"); + + logger.debug("The replica from which I expected the state, sent one which doesn't match the hash of the others, or it never sent it at all"); + + reset(); + requestState(); + + if (stateTimer != null) { + stateTimer.cancel(); + } + } + } + } + } + lockTimer.unlock(); + } + +} diff --git a/src/bftsmart/statemanagement/strategy/durability/StateSender.java b/src/bftsmart/statemanagement/durability/StateSender.java similarity index 96% rename from src/bftsmart/statemanagement/strategy/durability/StateSender.java rename to src/bftsmart/statemanagement/durability/StateSender.java index 7d6b24a7b..724045655 100644 --- a/src/bftsmart/statemanagement/strategy/durability/StateSender.java +++ b/src/bftsmart/statemanagement/durability/StateSender.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.statemanagement.strategy.durability; +package bftsmart.statemanagement.durability; import java.io.IOException; import java.io.ObjectOutputStream; diff --git a/src/bftsmart/statemanagement/strategy/durability/StateSenderServer.java b/src/bftsmart/statemanagement/durability/StateSenderServer.java similarity index 93% rename from src/bftsmart/statemanagement/strategy/durability/StateSenderServer.java rename to src/bftsmart/statemanagement/durability/StateSenderServer.java index 1518f9839..3f0b229f1 100644 --- a/src/bftsmart/statemanagement/strategy/durability/StateSenderServer.java +++ b/src/bftsmart/statemanagement/durability/StateSenderServer.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.statemanagement.strategy.durability; +package bftsmart.statemanagement.durability; import java.io.IOException; import java.net.ServerSocket; @@ -21,7 +21,7 @@ import bftsmart.statemanagement.ApplicationState; import bftsmart.tom.server.Recoverable; -import bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator; +import bftsmart.tom.server.durability.DurabilityCoordinator; import org.slf4j.Logger; import org.slf4j.LoggerFactory; diff --git a/src/bftsmart/statemanagement/strategy/StandardSMMessage.java b/src/bftsmart/statemanagement/standard/StandardSMMessage.java similarity index 97% rename from src/bftsmart/statemanagement/strategy/StandardSMMessage.java rename to src/bftsmart/statemanagement/standard/StandardSMMessage.java index e61f23eb1..def7f7f89 100644 --- a/src/bftsmart/statemanagement/strategy/StandardSMMessage.java +++ b/src/bftsmart/statemanagement/standard/StandardSMMessage.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.statemanagement.strategy; +package bftsmart.statemanagement.standard; import java.io.IOException; import java.io.ObjectInput; diff --git a/src/bftsmart/statemanagement/strategy/StandardStateManager.java b/src/bftsmart/statemanagement/standard/StandardStateManager.java similarity index 64% rename from src/bftsmart/statemanagement/strategy/StandardStateManager.java rename to src/bftsmart/statemanagement/standard/StandardStateManager.java index 7998b8698..f2c9ed29f 100644 --- a/src/bftsmart/statemanagement/strategy/StandardStateManager.java +++ b/src/bftsmart/statemanagement/standard/StandardStateManager.java @@ -1,20 +1,21 @@ /** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.statemanagement.strategy; + * Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package bftsmart.statemanagement.standard; +import bftsmart.statemanagement.StateManager; import java.util.ArrayList; import java.util.Arrays; import java.util.List; @@ -41,12 +42,12 @@ import org.slf4j.LoggerFactory; /** - * + * * @author Marcel Santos * */ -public class StandardStateManager extends BaseStateManager { - +public class StandardStateManager extends StateManager { + private Logger logger = LoggerFactory.getLogger(this.getClass()); private int replica; @@ -54,43 +55,26 @@ public class StandardStateManager extends BaseStateManager { private Timer stateTimer = null; private final static long INIT_TIMEOUT = 40000; private long timeout = INIT_TIMEOUT; - - //private LCManager lcManager; - private ExecutionManager execManager; - @Override public void init(TOMLayer tomLayer, DeliveryThread dt) { - SVController = tomLayer.controller; - - this.tomLayer = tomLayer; - this.dt = dt; - //this.lcManager = tomLayer.getSyncher().getLCManager(); - this.execManager = tomLayer.execManager; - changeReplica(); // initialize replica from which to ask the complete state - - //this.replica = 0; - //if (SVController.getCurrentViewN() > 1 && replica == SVController.getStaticConf().getProcessId()) - // changeReplica(); + super.init(tomLayer, dt); - state = null; - lastCID = -1; - waitingCID = -1; + changeReplica(); // initialize replica from which to ask the complete state - appStateOnly = false; } - + private void changeReplica() { - + int[] processes = this.SVController.getCurrentViewOtherAcceptors(); Random r = new Random(); - + int pos; do { //pos = this.SVController.getCurrentViewPos(replica); //replica = this.SVController.getCurrentViewProcesses()[(pos + 1) % SVController.getCurrentViewN()]; - + if (processes != null && processes.length > 1) { pos = r.nextInt(processes.length); replica = processes[pos]; @@ -100,71 +84,72 @@ private void changeReplica() { } } while (replica == SVController.getStaticConf().getProcessId()); } - + @Override protected void requestState() { - if (tomLayer.requestsTimer != null) - tomLayer.requestsTimer.clearAll(); + if (tomLayer.requestsTimer != null) { + tomLayer.requestsTimer.clearAll(); + } changeReplica(); // always ask the complete state to a different replica - + SMMessage smsg = new StandardSMMessage(SVController.getStaticConf().getProcessId(), waitingCID, TOMUtil.SM_REQUEST, replica, null, null, -1, -1); tomLayer.getCommunication().send(SVController.getCurrentViewOtherAcceptors(), smsg); logger.info("I just sent a request to the other replicas for the state up to CID " + waitingCID); - TimerTask stateTask = new TimerTask() { + TimerTask stateTask = new TimerTask() { public void run() { - logger.info("Timeout to retrieve state"); - int[] myself = new int[1]; - myself[0] = SVController.getStaticConf().getProcessId(); - tomLayer.getCommunication().send(myself, new StandardSMMessage(-1, waitingCID, TOMUtil.TRIGGER_SM_LOCALLY, -1, null, null, -1, -1)); + logger.info("Timeout to retrieve state"); + StandardSMMessage msg = new StandardSMMessage(-1, waitingCID, TOMUtil.TRIGGER_SM_LOCALLY, -1, null, null, -1, -1); + triggerTimeout(msg); } }; stateTimer = new Timer("state timer"); timeout = timeout * 2; - stateTimer.schedule(stateTask,timeout); + stateTimer.schedule(stateTask, timeout); } @Override public void stateTimeout() { lockTimer.lock(); logger.debug("Timeout for the replica that was supposed to send the complete state. Changing desired replica."); - if (stateTimer != null) - stateTimer.cancel(); + if (stateTimer != null) { + stateTimer.cancel(); + } changeReplica(); reset(); requestState(); lockTimer.unlock(); } - - @Override + + @Override public void SMRequestDeliver(SMMessage msg, boolean isBFT) { if (SVController.getStaticConf().isStateTransferEnabled() && dt.getRecoverer() != null) { - StandardSMMessage stdMsg = (StandardSMMessage)msg; + StandardSMMessage stdMsg = (StandardSMMessage) msg; boolean sendState = stdMsg.getReplica() == SVController.getStaticConf().getProcessId(); - + ApplicationState thisState = dt.getRecoverer().getState(msg.getCID(), sendState); if (thisState == null) { - + logger.warn("For some reason, I am sending a void state"); - thisState = dt.getRecoverer().getState(-1, sendState); + thisState = dt.getRecoverer().getState(-1, sendState); } - int[] targets = { msg.getSender() }; + int[] targets = {msg.getSender()}; SMMessage smsg = new StandardSMMessage(SVController.getStaticConf().getProcessId(), msg.getCID(), TOMUtil.SM_REPLY, -1, thisState, SVController.getCurrentView(), tomLayer.getSynchronizer().getLCManager().getLastReg(), tomLayer.execManager.getCurrentLeader()); - + logger.info("Sending state..."); tomLayer.getCommunication().send(targets, smsg); logger.info("Sent"); } } - @Override + @Override public void SMReplyDeliver(SMMessage msg, boolean isBFT) { lockTimer.lock(); if (SVController.getStaticConf().isStateTransferEnabled()) { @@ -173,27 +158,37 @@ public void SMReplyDeliver(SMMessage msg, boolean isBFT) { int currentLeader = -1; View currentView = null; CertifiedDecision currentProof = null; - + if (!appStateOnly) { - senderRegencies.put(msg.getSender(), msg.getRegency()); - senderLeaders.put(msg.getSender(), msg.getLeader()); - senderViews.put(msg.getSender(), msg.getView()); - senderProofs.put(msg.getSender(), msg.getState().getCertifiedDecision(SVController)); - if (enoughRegencies(msg.getRegency())) currentRegency = msg.getRegency(); - if (enoughLeaders(msg.getLeader())) currentLeader = msg.getLeader(); - if (enoughViews(msg.getView())) currentView = msg.getView(); - if (enoughProofs(waitingCID, this.tomLayer.getSynchronizer().getLCManager())) currentProof = msg.getState().getCertifiedDecision(SVController); - + senderRegencies.put(msg.getSender(), msg.getRegency()); + senderLeaders.put(msg.getSender(), msg.getLeader()); + senderViews.put(msg.getSender(), msg.getView()); + senderProofs.put(msg.getSender(), msg.getState().getCertifiedDecision(SVController)); + if (enoughRegencies(msg.getRegency())) { + currentRegency = msg.getRegency(); + } + if (enoughLeaders(msg.getLeader())) { + currentLeader = msg.getLeader(); + } + if (enoughViews(msg.getView())) { + currentView = msg.getView(); + } + if (enoughProofs(waitingCID, this.tomLayer.getSynchronizer().getLCManager())) { + currentProof = msg.getState().getCertifiedDecision(SVController); + } + } else { currentLeader = tomLayer.execManager.getCurrentLeader(); currentRegency = tomLayer.getSynchronizer().getLCManager().getLastReg(); currentView = SVController.getCurrentView(); } - + if (msg.getSender() == replica && msg.getState().getSerializedState() != null) { logger.debug("Expected replica sent state. Setting it to state"); state = msg.getState(); - if (stateTimer != null) stateTimer.cancel(); + if (stateTimer != null) { + stateTimer.cancel(); + } } senderStates.put(msg.getSender(), msg.getState()); @@ -203,33 +198,35 @@ public void SMReplyDeliver(SMMessage msg, boolean isBFT) { logger.debug("More than F confirmed"); ApplicationState otherReplicaState = getOtherReplicaState(); int haveState = 0; - if(state != null) { - byte[] hash = null; - hash = tomLayer.computeHash(state.getSerializedState()); - if (otherReplicaState != null) { - if (Arrays.equals(hash, otherReplicaState.getStateHash())) haveState = 1; - else if (getNumEqualStates() > SVController.getCurrentViewF()) - haveState = -1; + if (state != null) { + byte[] hash = null; + hash = tomLayer.computeHash(state.getSerializedState()); + if (otherReplicaState != null) { + if (Arrays.equals(hash, otherReplicaState.getStateHash())) { + haveState = 1; + } else if (getNumEqualStates() > SVController.getCurrentViewF()) { + haveState = -1; } } - - if (otherReplicaState != null && haveState == 1 && currentRegency > -1 && - currentLeader > -1 && currentView != null && (!isBFT || currentProof != null || appStateOnly)) { + } + + if (otherReplicaState != null && haveState == 1 && currentRegency > -1 + && currentLeader > -1 && currentView != null && (!isBFT || currentProof != null || appStateOnly)) { + + logger.info("Received state. Will install it"); - logger.info("Received state. Will install it"); - tomLayer.getSynchronizer().getLCManager().setLastReg(currentRegency); tomLayer.getSynchronizer().getLCManager().setNextReg(currentRegency); tomLayer.getSynchronizer().getLCManager().setNewLeader(currentLeader); tomLayer.execManager.setNewLeader(currentLeader); - + if (currentProof != null && !appStateOnly) { - + logger.debug("Installing proof for consensus " + waitingCID); Consensus cons = execManager.getConsensus(waitingCID); Epoch e = null; - + for (ConsensusMessage cm : currentProof.getConsMessages()) { e = cons.getEpoch(cm.getEpoch(), true, SVController); @@ -242,23 +239,20 @@ else if (getNumEqualStates() > SVController.getCurrentViewF()) if (cm.getType() == MessageFactory.ACCEPT) { e.setAccept(cm.getSender(), cm.getValue()); - } - - else if (cm.getType() == MessageFactory.WRITE) { + } else if (cm.getType() == MessageFactory.WRITE) { e.setWrite(cm.getSender(), cm.getValue()); } } - - + if (e != null) { byte[] hash = tomLayer.computeHash(currentProof.getDecision()); e.propValueHash = hash; e.propValue = currentProof.getDecision(); e.deserializedPropValue = tomLayer.checkProposedValue(currentProof.getDecision(), false); - cons.decided(e, false); - + cons.decided(e, false); + logger.info("Successfully installed proof for consensus " + waitingCID); } else { @@ -267,36 +261,39 @@ else if (cm.getType() == MessageFactory.WRITE) { } } - + // I might have timed out before invoking the state transfer, so // stop my re-transmission of STOP messages for all regencies up to the current one - if (currentRegency > 0) tomLayer.getSynchronizer().removeSTOPretransmissions(currentRegency - 1); + if (currentRegency > 0) { + tomLayer.getSynchronizer().removeSTOPretransmissions(currentRegency - 1); + } //if (currentRegency > 0) // tomLayer.requestsTimer.setTimeout(tomLayer.requestsTimer.getTimeout() * (currentRegency * 2)); - + dt.deliverLock(); waitingCID = -1; dt.update(state); - + if (!appStateOnly && execManager.stopped()) { Queue stoppedMsgs = execManager.getStoppedMsgs(); for (ConsensusMessage stopped : stoppedMsgs) { - if (stopped.getNumber() > state.getLastCID() /*msg.getCID()*/) + if (stopped.getNumber() > state.getLastCID() /*msg.getCID()*/) { execManager.addOutOfContextMessage(stopped); + } } execManager.clearStopped(); execManager.restart(); } - + tomLayer.processOutOfContext(); - + if (SVController.getCurrentViewId() != currentView.getId()) { logger.info("Installing current view!"); SVController.reconfigureTo(currentView); } - - isInitializing = false; - + + isInitializing = false; + dt.canDeliver(); dt.deliverUnlock(); @@ -306,18 +303,22 @@ else if (cm.getType() == MessageFactory.WRITE) { tomLayer.requestsTimer.Enabled(true); tomLayer.requestsTimer.startTimer(); - if (stateTimer != null) stateTimer.cancel(); - + if (stateTimer != null) { + stateTimer.cancel(); + } + if (appStateOnly) { - appStateOnly = false; + appStateOnly = false; tomLayer.getSynchronizer().resumeLC(); } } else if (otherReplicaState == null && (SVController.getCurrentViewN() / 2) < getReplies()) { waitingCID = -1; reset(); - - if (stateTimer != null) stateTimer.cancel(); - + + if (stateTimer != null) { + stateTimer.cancel(); + } + if (appStateOnly) { requestState(); } @@ -328,12 +329,16 @@ else if (cm.getType() == MessageFactory.WRITE) { reset(); requestState(); - if (stateTimer != null) stateTimer.cancel(); + if (stateTimer != null) { + stateTimer.cancel(); + } } else if (haveState == 0 && (SVController.getCurrentViewN() - SVController.getCurrentViewF()) <= getReplies()) { logger.debug("Could not obtain the state, retrying"); reset(); - if (stateTimer != null) stateTimer.cancel(); + if (stateTimer != null) { + stateTimer.cancel(); + } waitingCID = -1; //requestState(); } else { @@ -345,48 +350,44 @@ else if (cm.getType() == MessageFactory.WRITE) { } lockTimer.unlock(); } - + /** - * Search in the received states table for a state that was not sent by the expected - * replica. This is used to compare both states after received the state from expected - * and other replicas. + * Search in the received states table for a state that was not sent by the + * expected replica. This is used to compare both states after received the + * state from expected and other replicas. + * * @return The state sent from other replica */ private ApplicationState getOtherReplicaState() { - int[] processes = SVController.getCurrentViewProcesses(); - for(int process : processes) { - if(process == replica) - continue; - else { - ApplicationState otherState = senderStates.get(process); - if(otherState != null) - return otherState; - } - } - return null; + int[] processes = SVController.getCurrentViewProcesses(); + for (int process : processes) { + if (process == replica) { + continue; + } else { + ApplicationState otherState = senderStates.get(process); + if (otherState != null) { + return otherState; + } + } + } + return null; } private int getNumEqualStates() { - List states = new ArrayList(receivedStates()); - int match = 0; + List states = new ArrayList(receivedStates()); + int match = 0; for (ApplicationState st1 : states) { - int count = 0; + int count = 0; for (ApplicationState st2 : states) { - if(st1 != null && st1.equals(st2)) - count++; + if (st1 != null && st1.equals(st2)) { + count++; + } + } + if (count > match) { + match = count; } - if(count > match) - match = count; } return match; } - @Override - public void currentConsensusIdAsked(int sender) { - int me = SVController.getStaticConf().getProcessId(); - int lastConsensusId = tomLayer.getLastExec(); - SMMessage currentCID = new StandardSMMessage(me, lastConsensusId, TOMUtil.SM_REPLY_INITIAL, 0, null, null, 0, 0); - tomLayer.getCommunication().send(new int[]{sender}, currentCID); - } - } diff --git a/src/bftsmart/statemanagement/strategy/BaseStateManager.java b/src/bftsmart/statemanagement/strategy/BaseStateManager.java deleted file mode 100644 index f11d9851c..000000000 --- a/src/bftsmart/statemanagement/strategy/BaseStateManager.java +++ /dev/null @@ -1,289 +0,0 @@ -/** - * Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and - * the authors indicated in the @author tags - * - * Licensed under the Apache License, Version 2.0 (the "License"); you may not - * use this file except in compliance with the License. You may obtain a copy of - * the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - * License for the specific language governing permissions and limitations under - * the License. - */ -package bftsmart.statemanagement.strategy; - -import java.util.Collection; -import java.util.HashMap; -import java.util.Arrays; -import java.util.Set; - -import bftsmart.consensus.messages.ConsensusMessage; -import bftsmart.reconfiguration.ServerViewController; -import bftsmart.reconfiguration.views.View; -import bftsmart.statemanagement.ApplicationState; -import bftsmart.statemanagement.SMMessage; -import bftsmart.statemanagement.StateManager; -import bftsmart.tom.core.DeliveryThread; -import bftsmart.tom.core.TOMLayer; -import bftsmart.tom.leaderchange.LCManager; -import bftsmart.tom.leaderchange.CertifiedDecision; -import bftsmart.tom.util.TOMUtil; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -/** - * - * @author Marcel Santos - * - */ -public abstract class BaseStateManager implements StateManager { - - private Logger logger = LoggerFactory.getLogger(this.getClass()); - - protected TOMLayer tomLayer; - protected ServerViewController SVController; - protected DeliveryThread dt; - - protected HashMap senderStates = null; - protected HashMap senderViews = null; - protected HashMap senderRegencies = null; - protected HashMap senderLeaders = null; - protected HashMap senderProofs = null; - - protected boolean appStateOnly; - protected int waitingCID = -1; - protected int lastCID; - protected ApplicationState state; - - protected boolean isInitializing = true; - private HashMap senderCIDs = null; - - public BaseStateManager() { - senderStates = new HashMap<>(); - senderViews = new HashMap<>(); - senderRegencies = new HashMap<>(); - senderLeaders = new HashMap<>(); - senderProofs = new HashMap<>(); - } - - protected int getReplies() { - return senderStates.size(); - } - - protected boolean enoughReplies() { - return senderStates.size() > SVController.getCurrentViewF(); - } - - protected boolean enoughRegencies(int regency) { - return senderRegencies.size() > SVController.getQuorum(); - } - - protected boolean enoughLeaders(int leader) { - return senderLeaders.size() > SVController.getQuorum(); - } - - protected boolean enoughViews(View view) { - Collection views = senderViews.values(); - int counter = 0; - for (View v : views) { - if (view.equals(v)) { - counter++; - } - } - boolean result = counter > SVController.getQuorum(); - return result; - } - - // check if the consensus messages are consistent without checking the mac/signatures - // if it is consistent, it returns the respective consensus ID; otherwise, returns -1 - private int proofIsConsistent(Set proof) { - - int id = -1; - byte[] value = null; - - for (ConsensusMessage cm : proof) { - - if (id == -1) id = cm.getNumber(); - if (value == null) value = cm.getValue(); - - if (id != cm.getNumber() || !Arrays.equals(value, cm.getValue())) { - return -1; // they are not consistent, so the proof is invalid - } - - } - - // if the values are still these, this means the proof is empty, thus is invalid - if (id == -1 || value == null) return -1; - - return id; - } - - protected boolean enoughProofs(int cid, LCManager lc) { - - int counter = 0; - for (CertifiedDecision cDec : senderProofs.values()) { - - if (cDec != null && cid == proofIsConsistent(cDec.getConsMessages()) && lc.hasValidProof(cDec)) { - counter++; - } - - } - boolean result = counter > SVController.getQuorum(); - return result; - } - - /** - * Clear the collections and state hold by this object. Calls clear() in the - * States, Leaders, regencies and Views collections. Sets the state to - * null; - */ - protected void reset() { - senderStates.clear(); - senderLeaders.clear(); - senderRegencies.clear(); - senderViews.clear(); - senderProofs.clear(); - state = null; - } - - public Collection receivedStates() { - return senderStates.values(); - } - - @Override - public void setLastCID(int cid) { - lastCID = cid; - } - - @Override - public int getLastCID() { - return lastCID; - } - - @Override - public void requestAppState(int cid) { - lastCID = cid + 1; - waitingCID = cid; - logger.debug("Updated waitingcid to " + cid); - appStateOnly = true; - requestState(); - } - - @Override - public void analyzeState(int cid) { - logger.debug("The state transfer protocol is enabled"); - if (waitingCID == -1) { - logger.debug("I'm not waiting for any state, so I will keep record of this message"); - if (tomLayer.execManager.isDecidable(cid)) { - logger.info("I have now more than " + SVController.getCurrentViewF() + " messages for CID " + cid + " which are beyond CID " + lastCID); - lastCID = cid; - waitingCID = cid - 1; - logger.info("I will be waiting for state messages associated to consensus " + waitingCID); - requestState(); - } - } - } - - @Override - public abstract void init(TOMLayer tomLayer, DeliveryThread dt); - - @Override - public boolean isRetrievingState() { - if (isInitializing) { - return true; - } - return waitingCID > -1; - } - - @Override - public void askCurrentConsensusId() { - int me = SVController.getStaticConf().getProcessId(); - int[] target = SVController.getCurrentViewAcceptors(); - - SMMessage currentCID = new StandardSMMessage(me, -1, TOMUtil.SM_ASK_INITIAL, 0, null, null, 0, 0); - tomLayer.getCommunication().send(target, currentCID); - - target = SVController.getCurrentViewOtherAcceptors(); - - while (isInitializing) { - tomLayer.getCommunication().send(target, currentCID); - try { - Thread.sleep(1000); - } catch (InterruptedException e) { - logger.error("Interruption during sleep",e); - } - } - } - - @Override - public void currentConsensusIdAsked(int sender) { - int me = SVController.getStaticConf().getProcessId(); - int lastConsensusId = tomLayer.getLastExec(); - SMMessage currentCIDReply = new StandardSMMessage(me, lastConsensusId, TOMUtil.SM_REPLY_INITIAL, 0, null, null, 0, 0); - tomLayer.getCommunication().send(new int[]{sender}, currentCIDReply); - } - - @Override - public synchronized void currentConsensusIdReceived(SMMessage smsg) { - if (!isInitializing || waitingCID > -1) { - return; - } - if (senderCIDs == null) { - senderCIDs = new HashMap<>(); - } - senderCIDs.put(smsg.getSender(), smsg.getCID()); - if (senderCIDs.size() >= SVController.getQuorum()) { - - HashMap cids = new HashMap<>(); - for (int id : senderCIDs.keySet()) { - - int value = senderCIDs.get(id); - - Integer count = cids.get(value); - if (count == null) { - cids.put(value, 0); - } else { - cids.put(value, count + 1); - } - } - for (int key : cids.keySet()) { - if (cids.get(key) >= SVController.getQuorum()) { - if (key == lastCID) { - logger.info("Replica state is up to date"); - dt.deliverLock(); - isInitializing = false; - tomLayer.setLastExec(key); - dt.canDeliver(); - dt.deliverUnlock(); - break; - } else { - //ask for state - logger.info("Requesting state from other replicas"); - lastCID = key + 1; - if (waitingCID == -1) { - waitingCID = key; - requestState(); - } - } - } - } - } - } - - protected abstract void requestState(); - - @Override - public abstract void stateTimeout(); - - @Override - public abstract void SMRequestDeliver(SMMessage msg, boolean isBFT); - - @Override - public abstract void SMReplyDeliver(SMMessage msg, boolean isBFT); - -} diff --git a/src/bftsmart/statemanagement/strategy/durability/DurableStateManager.java b/src/bftsmart/statemanagement/strategy/durability/DurableStateManager.java deleted file mode 100644 index 17a0e6447..000000000 --- a/src/bftsmart/statemanagement/strategy/durability/DurableStateManager.java +++ /dev/null @@ -1,438 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. - */ -package bftsmart.statemanagement.strategy.durability; - -import java.io.IOException; -import java.io.ObjectInputStream; -import java.net.InetSocketAddress; -import java.net.Socket; -import java.net.UnknownHostException; -import java.util.Arrays; -import java.util.Queue; -import java.util.Timer; -import java.util.TimerTask; -import java.util.concurrent.locks.ReentrantLock; - -import bftsmart.tom.core.ExecutionManager; -import bftsmart.consensus.messages.ConsensusMessage; -import bftsmart.reconfiguration.views.View; -import bftsmart.statemanagement.ApplicationState; -import bftsmart.statemanagement.SMMessage; -import bftsmart.statemanagement.strategy.BaseStateManager; -import bftsmart.tom.core.DeliveryThread; -import bftsmart.tom.core.TOMLayer; -import bftsmart.tom.server.defaultservices.CommandsInfo; -import bftsmart.tom.server.defaultservices.durability.DurabilityCoordinator; -import bftsmart.tom.util.TOMUtil; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -public class DurableStateManager extends BaseStateManager { - - private Logger logger = LoggerFactory.getLogger(this.getClass()); - - //private LCManager lcManager; - private ExecutionManager execManager; - - private ReentrantLock lockTimer = new ReentrantLock(); - private Timer stateTimer = null; - private final static long INIT_TIMEOUT = 40000; - private long timeout = INIT_TIMEOUT; - - private CSTRequestF1 cstRequest; - - private CSTState stateCkp; - private CSTState stateLower; - private CSTState stateUpper; - - @Override - public void init(TOMLayer tomLayer, DeliveryThread dt) { - SVController = tomLayer.controller; - - this.tomLayer = tomLayer; - this.dt = dt; - //this.lcManager = tomLayer.getSyncher().getLCManager(); - this.execManager = tomLayer.execManager; - - state = null; - lastCID = 1; - waitingCID = -1; - - appStateOnly = false; - } - - @Override - protected void requestState() { - if (tomLayer.requestsTimer != null) - tomLayer.requestsTimer.clearAll(); - - int myProcessId = SVController.getStaticConf().getProcessId(); - int[] otherProcesses = SVController.getCurrentViewOtherAcceptors(); - int globalCkpPeriod = SVController.getStaticConf() - .getGlobalCheckpointPeriod(); - - CSTRequestF1 cst = new CSTRequestF1(waitingCID); - cst.defineReplicas(otherProcesses, globalCkpPeriod, myProcessId); - this.cstRequest = cst; - CSTSMMessage cstMsg = new CSTSMMessage(myProcessId, waitingCID, - TOMUtil.SM_REQUEST, cst, null, null, -1, -1); - tomLayer.getCommunication().send( - SVController.getCurrentViewOtherAcceptors(), cstMsg); - - logger.info("I just sent a request to the other replicas for the state up to CID " - + waitingCID); - - TimerTask stateTask = new TimerTask() { - public void run() { - int[] myself = new int[1]; - myself[0] = SVController.getStaticConf().getProcessId(); - tomLayer.getCommunication().send( - myself, - new CSTSMMessage(-1, waitingCID, - TOMUtil.TRIGGER_SM_LOCALLY, null, null, null, - -1, -1)); - } - }; - - stateTimer = new Timer("state timer"); - timeout = timeout * 2; - stateTimer.schedule(stateTask, timeout); - } - - @Override - public void stateTimeout() { - lockTimer.lock(); - logger.debug("(StateManager.stateTimeout) Timeout for the replica that was supposed to send the complete state. Changing desired replica."); - if (stateTimer != null) - stateTimer.cancel(); - reset(); - requestState(); - lockTimer.unlock(); - } - - @Override - public void SMRequestDeliver(SMMessage msg, boolean isBFT) { - logger.debug("Invoked method"); - if (SVController.getStaticConf().isStateTransferEnabled() - && dt.getRecoverer() != null) { - logger.debug("The state transfer protocol is enabled"); - logger.debug("I received a state request for CID " - + msg.getCID() + " from replica " + msg.getSender()); - CSTSMMessage cstMsg = (CSTSMMessage) msg; - CSTRequestF1 cstConfig = cstMsg.getCstConfig(); - boolean sendState = cstConfig.getCheckpointReplica() == SVController - .getStaticConf().getProcessId(); - if (sendState) - logger.debug("I should be the one sending the state"); - - logger.info("State asked by replica " + msg.getSender()); - - int[] targets = { msg.getSender() }; - InetSocketAddress address = SVController.getCurrentView().getAddress( - SVController.getStaticConf().getProcessId()); - String myIp = address.getHostName(); - int myId = SVController.getStaticConf().getProcessId(); - int port = 4444 + myId; - address = new InetSocketAddress(myIp, port); - cstConfig.setAddress(address); - CSTSMMessage reply = new CSTSMMessage(myId, msg.getCID(), - TOMUtil.SM_REPLY, cstConfig, null, - SVController.getCurrentView(), tomLayer.getSynchronizer().getLCManager().getLastReg(), - tomLayer.execManager.getCurrentLeader()); - - StateSenderServer stateServer = new StateSenderServer(port); - stateServer.setRecoverable(dt.getRecoverer()); - stateServer.setRequest(cstConfig); - new Thread(stateServer).start(); - - tomLayer.getCommunication().send(targets, reply); - - } - } - - @Override - public void SMReplyDeliver(SMMessage msg, boolean isBFT) { - lockTimer.lock(); - CSTSMMessage reply = (CSTSMMessage) msg; - if (SVController.getStaticConf().isStateTransferEnabled()) { - logger.debug("The state transfer protocol is enabled"); - logger.info("I received a state reply for CID " - + reply.getCID() - + " from replica " - + reply.getSender()); - - logger.info("Received CID: " + reply.getCID() - + ". Waiting " + waitingCID); - if (waitingCID != -1 && reply.getCID() == waitingCID) { - - int currentRegency = -1; - int currentLeader = -1; - View currentView = null; - // CertifiedDecision currentProof = null; - - if (!appStateOnly) { - senderRegencies.put(reply.getSender(), reply.getRegency()); - senderLeaders.put(reply.getSender(), reply.getLeader()); - senderViews.put(reply.getSender(), reply.getView()); - // senderProofs.put(msg.getSender(), msg.getState().getCertifiedDecision(SVController)); - if (enoughRegencies(reply.getRegency())) - currentRegency = reply.getRegency(); - if (enoughLeaders(reply.getLeader())) - currentLeader = reply.getLeader(); - if (enoughViews(reply.getView())) { - currentView = reply.getView(); - if (!currentView.isMember(SVController.getStaticConf() - .getProcessId())) { - logger.warn("Not a member!"); - } - } - // if (enoughProofs(waitingCID, this.tomLayer.getSynchronizer().getLCManager())) currentProof = msg.getState().getCertifiedDecision(SVController); - - } else { - currentLeader = tomLayer.execManager.getCurrentLeader(); - currentRegency = tomLayer.getSynchronizer().getLCManager().getLastReg(); - currentView = SVController.getCurrentView(); - } - - logger.debug("The reply is for the CID that I want!"); - - InetSocketAddress address = reply.getCstConfig().getAddress(); - Socket clientSocket; - ApplicationState stateReceived = null; - try { - clientSocket = new Socket(address.getHostName(), - address.getPort()); - ObjectInputStream in = new ObjectInputStream( - clientSocket.getInputStream()); - stateReceived = (ApplicationState) in.readObject(); - } catch (UnknownHostException e) { - // TODO Auto-generated catch block - logger.error("Failed to connect to address",e); - } catch (IOException e) { - // TODO Auto-generated catch block - logger.error("Failed to connect to address",e); - } catch (ClassNotFoundException e) { - // TODO Auto-generated catch block - logger.error("Failed to deserialize application state object",e); - } - - if (stateReceived instanceof CSTState) { - senderStates.put(reply.getSender(), stateReceived); - if (reply.getSender() == cstRequest.getCheckpointReplica()) - this.stateCkp = (CSTState) stateReceived; - if (reply.getSender() == cstRequest.getLogLower()) - this.stateLower = (CSTState) stateReceived; - if (reply.getSender() == cstRequest.getLogUpper()) - this.stateUpper = (CSTState) stateReceived; - } - - if (senderStates.size() == 3) { - - CommandsInfo[] lowerLog = stateLower.getLogLower(); - CommandsInfo[] upperLog = stateUpper.getLogUpper(); - logger.info("lowerLog "); - if (lowerLog != null) - logger.info("\t" + lowerLog.length); - logger.info("upperLog "); - if (upperLog != null) - logger.info("\t" + upperLog.length); - - boolean haveState = false; - byte[] lowerbytes = TOMUtil.getBytes(lowerLog); - logger.debug("Log lower bytes size: " - + lowerbytes.length); - byte[] upperbytes = TOMUtil.getBytes(upperLog); - logger.debug("Log upper bytes size: " - + upperbytes.length); - - byte[] lowerLogHash = TOMUtil.computeHash(lowerbytes); - byte[] upperLogHash = TOMUtil.computeHash(upperbytes); - - // validate lower log - if (Arrays.equals(stateCkp.getHashLogLower(), lowerLogHash)) - haveState = true; - else - logger.error("Lower log does not match"); - // validate upper log - if (!haveState || !Arrays.equals(stateCkp.getHashLogUpper(), upperLogHash)) { - haveState = false; - logger.error("Upper log does not match"); - } - - CSTState statePlusLower = new CSTState(stateCkp.getSerializedState(), - TOMUtil.getBytes(stateCkp.getSerializedState()), - stateLower.getLogLower(), stateCkp.getHashLogLower(), null, null, - stateCkp.getCheckpointCID(), stateUpper.getCheckpointCID(), SVController.getStaticConf().getProcessId()); - - if (haveState) { // validate checkpoint - logger.info("validating checkpoint!!!"); - dt.getRecoverer().setState(statePlusLower); - byte[] currentStateHash = ((DurabilityCoordinator) dt.getRecoverer()).getCurrentStateHash(); - if (!Arrays.equals(currentStateHash, stateUpper.getHashCheckpoint())) { - logger.error("ckp hash don't match"); - haveState = false; - } - } - - logger.info("Current regency: " + currentRegency); - logger.info("Current leader: " + currentLeader); - logger.info("Current view: " + currentView); - if (currentRegency > -1 && currentLeader > -1 - && currentView != null && haveState && (!isBFT || /*currentProof != null ||*/ appStateOnly)) { - logger.info("---- RECEIVED VALID STATE ----"); - - logger.debug("The state of those replies is good!"); - logger.debug("CID State requested: " + reply.getCID()); - logger.debug("CID State received: " + stateUpper.getLastCID()); - - tomLayer.getSynchronizer().getLCManager().setLastReg(currentRegency); - tomLayer.getSynchronizer().getLCManager().setNextReg(currentRegency); - tomLayer.getSynchronizer().getLCManager().setNewLeader(currentLeader); - - tomLayer.execManager.setNewLeader(currentLeader); - -// if (currentProof != null && !appStateOnly) { -// -// System.out.println("Installing proof for consensus " + waitingCID); -// -// Consensus cons = execManager.getConsensus(waitingCID); -// Epoch e = null; -// -// for (ConsensusMessage cm : currentProof.getConsMessages()) { -// -// e = cons.getEpoch(cm.getEpoch(), true, SVController); -// if (e.getTimestamp() != cm.getEpoch()) { -// -// System.out.println("Strange... proof contains messages from more than just one epoch"); -// e = cons.getEpoch(cm.getEpoch(), true, SVController); -// } -// e.addToProof(cm); -// -// if (cm.getType() == MessageFactory.ACCEPT) { -// e.setAccept(cm.getSender(), cm.getValue()); -// } -// -// else if (cm.getType() == MessageFactory.WRITE) { -// e.setWrite(cm.getSender(), cm.getValue()); -// } -// -// } -// -// -// if (e != null) { -// -// byte[] hash = tomLayer.computeHash(currentProof.getDecision()); -// e.propValueHash = hash; -// e.propValue = currentProof.getDecision(); -// e.deserializedPropValue = tomLayer.checkProposedValue(currentProof.getDecision(), false); -// cons.decided(e, false); -// -// System.out.println("Successfully installed proof for consensus " + waitingCID); -// -// } else { -// System.out.println("Failed to install proof for consensus " + waitingCID); -// -// } -// -// } - - - // I might have timed out before invoking the state transfer, so - // stop my re-transmission of STOP messages for all regencies up to the current one - if (currentRegency > 0) tomLayer.getSynchronizer().removeSTOPretransmissions(currentRegency - 1); - - logger.debug("trying to acquire deliverlock"); - dt.deliverLock(); - logger.debug("acquired"); - - // this makes the isRetrievingState() evaluates to false - waitingCID = -1; - dt.update(stateUpper); - - // Deal with stopped messages that may come from - // synchronization phase - if (!appStateOnly && execManager.stopped()) { - Queue stoppedMsgs = execManager.getStoppedMsgs(); - for (ConsensusMessage stopped : stoppedMsgs) { - if (stopped.getNumber() > state.getLastCID()) - execManager.addOutOfContextMessage(stopped); - } - execManager.clearStopped(); - execManager.restart(); - } - - logger.info("Processing out of context messages"); - tomLayer.processOutOfContext(); - - if (SVController.getCurrentViewId() != currentView.getId()) { - logger.info("Installing current view!"); - SVController.reconfigureTo(currentView); - } - - isInitializing = false; - - dt.canDeliver(); - dt.deliverUnlock(); - - reset(); - - logger.info("I updated the state!"); - - tomLayer.requestsTimer.Enabled(true); - tomLayer.requestsTimer.startTimer(); - if (stateTimer != null) - stateTimer.cancel(); - - if (appStateOnly) { - appStateOnly = false; - tomLayer.getSynchronizer().resumeLC(); - } - } else if (state == null - && (SVController.getCurrentViewN() / 2) < getReplies()) { - logger.warn("---- DIDNT RECEIVE STATE ----"); - - logger.debug("I have more than " - + (SVController.getCurrentViewN() / 2) - + " messages that are no good!"); - - waitingCID = -1; - reset(); - - if (stateTimer != null) - stateTimer.cancel(); - - if (appStateOnly) { - requestState(); - } - } else if (!haveState) { - logger.warn("---- RECEIVED INVALID STATE ----"); - - logger.debug("The replica from which I expected the state, sent one which doesn't match the hash of the others, or it never sent it at all"); - - reset(); - requestState(); - - if (stateTimer != null) - stateTimer.cancel(); - } - } - } - } - lockTimer.unlock(); - } - -} diff --git a/src/bftsmart/tom/AsynchServiceProxy.java b/src/bftsmart/tom/AsynchServiceProxy.java index 9f8d68402..f6ca06680 100644 --- a/src/bftsmart/tom/AsynchServiceProxy.java +++ b/src/bftsmart/tom/AsynchServiceProxy.java @@ -7,7 +7,6 @@ import bftsmart.tom.util.Extractor; import bftsmart.tom.util.KeyLoader; import bftsmart.tom.util.TOMUtil; -import java.security.Provider; import java.util.Arrays; import java.util.Comparator; import java.util.HashMap; @@ -19,48 +18,60 @@ * This class is an extension of 'ServiceProxy' that can waits for replies * asynchronously. * - * @author Andre Nogueira - * */ public class AsynchServiceProxy extends ServiceProxy { private Logger logger = LoggerFactory.getLogger(this.getClass()); - /** - * - */ private HashMap requestsContext; private HashMap requestsReplies; private HashMap requestsAlias; - /** +/** + * Constructor * - * @param processId Replica id + * @see bellow */ public AsynchServiceProxy(int processId) { this(processId, null); init(); } - /** +/** + * Constructor * - * @param processId Replica id - * @param configHome Configuration folder + * @see bellow */ public AsynchServiceProxy(int processId, String configHome) { super(processId, configHome); init(); } - - public AsynchServiceProxy(int processId, String configHome, KeyLoader loader, Provider provider) { - super(processId, configHome, loader, provider); + + /** + * Constructor + * + * @see bellow + */ + public AsynchServiceProxy(int processId, String configHome, KeyLoader loader) { + super(processId, configHome, loader); init(); } + /** + * Constructor + * + * @param processId Process id for this client (should be different from replicas) + * @param configHome Configuration directory for BFT-SMART + * @param replyComparator Used for comparing replies from different servers + * to extract one returned by f+1 + * @param replyExtractor Used for extracting the response from the matching + * quorum of replies + * @param loader Used to load signature keys from disk + */ public AsynchServiceProxy(int processId, String configHome, - Comparator replyComparator, Extractor replyExtractor, KeyLoader loader, Provider provider) { + Comparator replyComparator, Extractor replyExtractor, KeyLoader loader) { - super(processId, configHome, replyComparator, replyExtractor, loader, provider); + super(processId, configHome, replyComparator, replyExtractor, loader); init(); } @@ -76,31 +87,31 @@ private View newView(byte[] bytes) { return (o != null && o instanceof View ? (View) o : null); } /** - * - * @param request - * @param replyListener - * @param reqType Request type - * @return + * @see bellow */ public int invokeAsynchRequest(byte[] request, ReplyListener replyListener, TOMMessageType reqType) { return invokeAsynchRequest(request, super.getViewManager().getCurrentViewProcesses(), replyListener, reqType); } /** - * - * @param request - * @param targets - * @param replyListener + * This method asynchronously sends a request to the replicas. + * + * @param request Request to be sent + * @param targets The IDs for the replicas to which to send the request + * @param replyListener Callback object that handles reception of replies * @param reqType Request type - * @return + * + * @return A unique identification for the request */ public int invokeAsynchRequest(byte[] request, int[] targets, ReplyListener replyListener, TOMMessageType reqType) { return invokeAsynch(request, targets, replyListener, reqType); } /** - * - * @param requestId Request + * Purges all information associated to the request. + * This should always be invoked once enough replies are received and processed by the ReplyListener callback. + * + * @param requestId A unique identification for a previously sent request */ public void cleanAsynchRequest(int requestId) { @@ -118,7 +129,9 @@ public void cleanAsynchRequest(int requestId) { } /** + * This is the method invoked by the client side communication system. * + * @param reply The reply delivered by the client side communication system */ @Override public void replyReceived(TOMMessage reply) { @@ -206,14 +219,6 @@ public void run() { } } - /** - * - * @param request - * @param targets - * @param replyListener - * @param reqType - * @return - */ private int invokeAsynch(byte[] request, int[] targets, ReplyListener replyListener, TOMMessageType reqType) { logger.debug("Asynchronously sending request to " + Arrays.toString(targets)); diff --git a/src/bftsmart/tom/MessageContext.java b/src/bftsmart/tom/MessageContext.java index 00dd18a20..2964cddcb 100644 --- a/src/bftsmart/tom/MessageContext.java +++ b/src/bftsmart/tom/MessageContext.java @@ -28,7 +28,6 @@ * It stores all informations regarding the message sent by the client, as well as * the consensus instance in which it was ordered. * - * @author alysson */ public class MessageContext implements Serializable { @@ -64,23 +63,23 @@ public class MessageContext implements Serializable { /** * Constructor * - * @param sender - * @param viewID - * @param type - * @param session - * @param sequence - * @param operationId - * @param replyServer - * @param signature - * @param timestamp - * @param numOfNonces - * @param seed - * @param regency - * @param leader - * @param consensusId - * @param proof - * @param firstInBatch - * @param noOp + * @param sender The sender ID + * @param viewID The view ID + * @param type The request type + * @param session Client's session + * @param sequence Request's sequence number + * @param operationId The operation ID + * @param replyServer ID of the server expected to send a non-hashed reply when the clients sends an unordered hashed request. + * @param signature Client's signature. + * @param timestamp The timestamp as generated by the leader + * @param numOfNonces The nonces as generated by the leader + * @param seed The seed as generated by the leader + * @param regency The current regency in which the message was ordered. + * @param leader The leader with which the batch was decided + * @param consensusId The ID of the consensus in which this request was ordered + * @param proof The proof for the consensus + * @param firstInBatch The first request in the ordered batch + * @param noOp true if the consensus instance has no operations to deliver to the application */ public MessageContext(int sender, int viewID, TOMMessageType type, int session, int sequence, int operationId, int replyServer, byte[] signature, @@ -111,41 +110,65 @@ public MessageContext(int sender, int viewID, TOMMessageType type, } /** - * - * - * @return the view id + * Returns the view ID + * @return The view ID */ public int getViewID() { return viewID; } + /** + * Returns the request type + * @return The request type + */ public TOMMessageType getType() { return type; } + /** + * Returns the client's session + * @return Client's session + */ public int getSession() { return session; } + /** + * Returns the request's sequence number + * @return Request's sequence number + */ public int getSequence() { return sequence; } + /** + * Returns the operation ID + * @return The operation ID + */ public int getOperationId() { return operationId; } + /** + * Returns the ID of the server expected to send a non-hashed reply when the clients sends an unordered hashed request. + * @return ID of the server expected to send a non-hashed reply when the clients sends an unordered hashed request. + */ public int getReplyServer() { return replyServer; } + + /** + * Returns the client's signature. + * @return Client's signature. + */ public byte[] getSignature() { return signature; } /** - * Returns the id of the client that sent the message - * @return the sender id + * Returns the ID of the client that sent the message + * @return The sender ID */ public int getSender() { return sender; @@ -153,7 +176,7 @@ public int getSender() { /** * Returns the timestamp as generated by the leader - * @return the timestamp + * @return The timestamp as generated by the leader */ public long getTimestamp() { return timestamp; @@ -161,7 +184,7 @@ public long getTimestamp() { /** * Returns the nonces as generated by the leader - * @return the nonces + * @return The nonces as generated by the leader */ public byte[] getNonces() { @@ -188,16 +211,16 @@ public int getNumOfNonces() { /** * Returns the seed as generated by the leader - * @return the seed + * @return The seed as generated by the leader */ public long getSeed() { return seed; } /** - * Returns the id of the consensus in which this message was ordered (-1 + * Returns the ID of the consensus in which this request was ordered (-1 * if readonly). - * @return the consensus Id + * @return The ID of the consensus in which this request was ordered */ public int getConsensusId() { return consensusId; @@ -206,14 +229,14 @@ public int getConsensusId() { /** * Returns the id of the leader replica in which this message was ordered * (-1 if readonly). - * @return the leader with which the batch was decided + * @return The leader with which the batch was decided */ public int getLeader() { return leader; } /** * Returns the proof for the consensus. - * @return the proof for the consensus + * @return The proof for the consensus */ public Set getProof() { return proof; @@ -221,27 +244,41 @@ public Set getProof() { /** * Returns the current regency in which the message was ordered. - * @return the regency + * @return The current regency in which the message was ordered. */ public int getRegency() { return regency; } /** - * @return the first message in the ordered batch + * Returns the first request in the ordered batch + * + * @return The first request in the ordered batch */ public TOMMessage getFirstInBatch() { return firstInBatch; } - + + /** + * @deprecated + */ public void setLastInBatch() { lastInBatch = true; } + /** + * Returns the last request in the ordered batch + * + * @return The last request in the ordered batch + */ public boolean isLastInBatch() { return lastInBatch; } + /** + * Returns true if the consensus instance has no operations to deliver to the application + * @return true if the consensus instance has no operations to deliver to the application + */ public boolean isNoOp() { return noOp; } @@ -249,7 +286,9 @@ public boolean isNoOp() { /** * Generates a TOMMessage for its associated requests using the new info that it now supports since the previous commit. * It is assumed that the byte array passed to this method is the serialized request associated to the original TOMMessage. + * * @param content Serialized request associated to the original TOMMessage. + * * @return A TOMMessage object that is equal to the original object issued by the client */ public TOMMessage recreateTOMMessage(byte[] content) { diff --git a/src/bftsmart/tom/ReplicaContext.java b/src/bftsmart/tom/ReplicaContext.java index 4e51f7a56..ab1fcb901 100644 --- a/src/bftsmart/tom/ReplicaContext.java +++ b/src/bftsmart/tom/ReplicaContext.java @@ -23,13 +23,18 @@ /** * This class contains information related to the replica. * - * @author Alysson Bessani */ public class ReplicaContext { private ServerCommunicationSystem cs; // Server side comunication system private ServerViewController SVController; + /** + * Constructor + * + * @param cs + * @param SVController + */ public ReplicaContext(ServerCommunicationSystem cs, ServerViewController SVController) { this.cs = cs; @@ -65,11 +70,11 @@ public View getCurrentView() { return SVController.getCurrentView(); } - public ServerCommunicationSystem getServerCommunicationSystem() { - return cs; - } - - public void setServerCommunicationSystem(ServerCommunicationSystem cs) { - this.cs = cs; - } + /** + * Returns the replica's communication system + * @return The replica's communication system + */ + public ServerCommunicationSystem getServerCommunicationSystem() { + return cs; + } } diff --git a/src/bftsmart/tom/RequestContext.java b/src/bftsmart/tom/RequestContext.java index 6613d29dd..aa1bd30f6 100644 --- a/src/bftsmart/tom/RequestContext.java +++ b/src/bftsmart/tom/RequestContext.java @@ -17,6 +17,17 @@ public final class RequestContext{ private final byte[] request; + /** + * Constructor + * + * @param reqId Request ID + * @param operationId Operation ID + * @param requestType The request type + * @param targets IDs of the targets to which the request was sent + * @param sendingTime Sending time + * @param replyListener Reply listener associated to the request + * @param request Payload of the request + */ public RequestContext(int reqId, int operationId, TOMMessageType requestType, int [] targets, long sendingTime, ReplyListener replyListener, byte[] request) { this.reqId = reqId; @@ -28,24 +39,60 @@ public RequestContext(int reqId, int operationId, TOMMessageType requestType, in this.request = request; } + /** + * Returns the request unique ID + * + * @return Request ID + */ public final int getReqId() { return reqId; } + + /** + * Returns the operation ID + * + * @return Operation ID + */ public final int getOperationId() { return operationId; } + + /** + * Returns the request type + * @return The request type + */ public final TOMMessageType getRequestType() { return requestType; } + + /** + * Returns the sending time + * @return Sending time + */ public final long getSendingTime() { return sendingTime; } + + /** + * Returns the reply listener associated to the request + * @return Reply listener associated to the request + */ public ReplyListener getReplyListener(){ return replyListener; } + + /** + * Returns the IDs of the targets to which the request was sent + * @return IDs of the targets to which the request was sent + */ public int [] getTargets() { return targets; } + + /** + * The payload of the request + * @return Payload of the request + */ public byte [] getRequest() { return request; } diff --git a/src/bftsmart/tom/ServiceProxy.java b/src/bftsmart/tom/ServiceProxy.java index ff12a520f..bbb3b8820 100644 --- a/src/bftsmart/tom/ServiceProxy.java +++ b/src/bftsmart/tom/ServiceProxy.java @@ -15,6 +15,7 @@ */ package bftsmart.tom; +import bftsmart.tom.core.TOMSender; import java.util.Arrays; import java.util.Comparator; import java.util.Random; @@ -69,7 +70,7 @@ public class ServiceProxy extends TOMSender { * @see bellow */ public ServiceProxy(int processId) { - this(processId, null, null, null, null, null); + this(processId, null, null, null, null); } /** @@ -78,11 +79,16 @@ public ServiceProxy(int processId) { * @see bellow */ public ServiceProxy(int processId, String configHome) { - this(processId, configHome, null, null, null, null); + this(processId, configHome, null, null, null); } - public ServiceProxy(int processId, String configHome, KeyLoader loader, Provider provider) { - this(processId, configHome, null, null, loader, provider); + /** + * Constructor + * + * @see bellow + */ + public ServiceProxy(int processId, String configHome, KeyLoader loader) { + this(processId, configHome, null, null, loader); } /** @@ -90,17 +96,18 @@ public ServiceProxy(int processId, String configHome, KeyLoader loader, Provider * * @param processId Process id for this client (should be different from replicas) * @param configHome Configuration directory for BFT-SMART - * @param replyComparator used for comparing replies from different servers + * @param replyComparator Used for comparing replies from different servers * to extract one returned by f+1 - * @param replyExtractor used for extracting the response from the matching + * @param replyExtractor Used for extracting the response from the matching * quorum of replies + * @param loader Used to load signature keys from disk */ public ServiceProxy(int processId, String configHome, - Comparator replyComparator, Extractor replyExtractor, KeyLoader loader, Provider provider) { + Comparator replyComparator, Extractor replyExtractor, KeyLoader loader) { if (configHome == null) { - init(processId, loader, provider); + init(processId, loader); } else { - init(processId, configHome, loader, provider); + init(processId, configHome, loader); } replies = new TOMMessage[getViewManager().getCurrentViewN()]; @@ -125,12 +132,18 @@ public TOMMessage extractResponse(TOMMessage[] replies, int sameContent, int las * Get the amount of time (in seconds) that this proxy will wait for * servers replies before returning null. * - * @return the invokeTimeout + * @return the timeout value in seconds */ public int getInvokeTimeout() { return invokeTimeout; } + /** + * Get the amount of time (in seconds) that this proxy will wait for + * servers unordered hashed replies before returning null. + * + * @return the timeout value in seconds + */ public int getInvokeUnorderedHashedTimeout() { return invokeUnorderedHashedTimeout; } @@ -139,25 +152,57 @@ public int getInvokeUnorderedHashedTimeout() { * Set the amount of time (in seconds) that this proxy will wait for * servers replies before returning null. * - * @param invokeTimeout the invokeTimeout to set + * @param invokeTimeout the timeout value to set */ public void setInvokeTimeout(int invokeTimeout) { this.invokeTimeout = invokeTimeout; } + /** + * Set the amount of time (in seconds) that this proxy will wait for + * servers unordered hashed replies before returning null. + * + * @param timeout the timeout value to set + */ public void setInvokeUnorderedHashedTimeout(int timeout) { this.invokeUnorderedHashedTimeout = timeout; } + /** + * This method sends an ordered request to the replicas, and returns the related reply. + * If the servers take more than invokeTimeout seconds the method returns null. + * This method is thread-safe. + * + * @param request to be sent + * @return The reply from the replicas related to request + */ public byte[] invokeOrdered(byte[] request) { return invoke(request, TOMMessageType.ORDERED_REQUEST); } - public byte[] invokeUnordered(byte[] request) { + /** + * This method sends an unordered request to the replicas, and returns the related reply. + * If the servers take more than invokeTimeout seconds the method returns null. + * This method is thread-safe. + * + * @param request to be sent + * @return The reply from the replicas related to request + */ + public byte[] invokeUnordered(byte[] request) { return invoke(request, TOMMessageType.UNORDERED_REQUEST); } - public byte[] invokeUnorderedHashed(byte[] request) { + /** + * This method sends an unordered request to the replicas, and returns the related reply. + * This method chooses randomly one replica to send the complete response, while the others + * only send a hash of that response. + * If the servers take more than invokeTimeout seconds the method returns null. + * This method is thread-safe. + * + * @param request to be sent + * @return The reply from the replicas related to request + */ + public byte[] invokeUnorderedHashed(byte[] request) { return invoke(request, TOMMessageType.UNORDERED_HASHED_REQUEST); } @@ -167,11 +212,15 @@ public byte[] invokeUnorderedHashed(byte[] request) { * This method is thread-safe. * * @param request Request to be sent - * @param reqType TOM_NORMAL_REQUESTS for service requests, and other for - * reconfig requests. + * @param reqType ORDERED_REQUEST/UNORDERED_REQUEST/UNORDERED_HASHED_REQUEST for normal requests, and RECONFIG for + * reconfiguration requests. + * * @return The reply from the replicas related to request */ public byte[] invoke(byte[] request, TOMMessageType reqType) { + + try { + canSendLock.lock(); // Clean all statefull data to prepare for receiving next replies @@ -293,11 +342,18 @@ public byte[] invoke(byte[] request, TOMMessageType reqType) { } //******* EDUARDO END **************// - canSendLock.unlock(); return ret; + + } finally { + + if (canSendLock.isHeldByCurrentThread()) canSendLock.unlock(); //always release lock + } } //******* EDUARDO BEGIN **************// + /** + * @deprecated + */ protected void reconfigureTo(View v) { logger.debug("Installing a most up-to-date view with id=" + v.getId()); getViewManager().reconfigureTo(v); @@ -403,6 +459,11 @@ public void replyReceived(TOMMessage reply) { } } + /** + * Retrieves the required quorum size for the amount of replies + * + * @return The quorum size for the amount of replies + */ protected int getReplyQuorum() { if (getViewManager().getStaticConf().isBFT()) { return (int) Math.ceil((getViewManager().getCurrentViewN() diff --git a/src/bftsmart/tom/ServiceReplica.java b/src/bftsmart/tom/ServiceReplica.java index 9384f9e7d..7ede372e5 100644 --- a/src/bftsmart/tom/ServiceReplica.java +++ b/src/bftsmart/tom/ServiceReplica.java @@ -36,7 +36,6 @@ import bftsmart.tom.leaderchange.CertifiedDecision; import bftsmart.tom.server.BatchExecutable; import bftsmart.tom.server.Executable; -import bftsmart.tom.server.FIFOExecutable; import bftsmart.tom.server.Recoverable; import bftsmart.tom.server.Replier; import bftsmart.tom.server.RequestVerifier; @@ -88,7 +87,7 @@ public class ServiceReplica { * @param recoverer Recoverer */ public ServiceReplica(int id, Executable executor, Recoverable recoverer) { - this(id, "", executor, recoverer, null, new DefaultReplier(), null, null); + this(id, "", executor, recoverer, null, new DefaultReplier(), null); } /** @@ -100,38 +99,40 @@ public ServiceReplica(int id, Executable executor, Recoverable recoverer) { * @param verifier Requests verifier */ public ServiceReplica(int id, Executable executor, Recoverable recoverer, RequestVerifier verifier) { - this(id, "", executor, recoverer, verifier, new DefaultReplier(), null, null); + this(id, "", executor, recoverer, verifier, new DefaultReplier(), null); } /** * Constructor * - * @param id Replica ID - * @param executor Executor - * @param recoverer Recoverer - * @param verifier Requests verifier - * @param replier Replier + * @see bellow */ public ServiceReplica(int id, Executable executor, Recoverable recoverer, RequestVerifier verifier, Replier replier) { - this(id, "", executor, recoverer, verifier, replier, null, null); + this(id, "", executor, recoverer, verifier, replier, null); } + /** + * Constructor + * + * @see bellow + */ public ServiceReplica(int id, Executable executor, Recoverable recoverer, RequestVerifier verifier, Replier replier, KeyLoader loader, Provider provider) { - this(id, "", executor, recoverer, verifier, replier, loader, provider); + this(id, "", executor, recoverer, verifier, replier, loader); } /** * Constructor * - * @param id Process ID - * @param configHome Configuration directory for JBP - * @param executor Executor - * @param recoverer Recoverer - * @param verifier Requests verifier - * @param replier Replier + * @param id Replica ID + * @param configHome Configuration directory for BFT-SMART + * @param executor The executor implementation + * @param recoverer The recoverer implementation + * @param verifier Requests Verifier + * @param replier Can be used to override the targets of the replies associated to each request. + * @param loader Used to load signature keys from disk */ - public ServiceReplica(int id, String configHome, Executable executor, Recoverable recoverer, RequestVerifier verifier, Replier replier, KeyLoader loader, Provider provider) { + public ServiceReplica(int id, String configHome, Executable executor, Recoverable recoverer, RequestVerifier verifier, Replier replier, KeyLoader loader) { this.id = id; - this.SVController = new ServerViewController(id, configHome, loader, provider); + this.SVController = new ServerViewController(id, configHome, loader); this.executor = executor; this.recoverer = recoverer; this.replier = (replier != null ? replier : new DefaultReplier()); @@ -141,10 +142,6 @@ public ServiceReplica(int id, String configHome, Executable executor, Recoverabl this.replier.setReplicaContext(replicaCtx); } - public void setReplyController(Replier replier) { - this.replier = replier; - } - // this method initializes the object private void init() { try { @@ -173,6 +170,10 @@ private void init() { initReplica(); } + /** + * + * @deprecated + */ public void joinMsgReceived(VMMessage msg) { ReconfigureReply r = msg.getReply(); @@ -194,41 +195,33 @@ private void initReplica() { } /** - * This message delivers a readonly message, i.e., a message that was not - * ordered to the replica and gather the reply to forward to the client - * - * @param message the request received from the delivery thread - * @param msgCtx the context for the message + * @deprecated */ public final void receiveReadonlyMessage(TOMMessage message, MessageContext msgCtx) { - byte[] response; + TOMMessage response; // This is used to deliver the requests to the application and obtain a reply to deliver //to the clients. The raw decision does not need to be delivered to the recoverable since // it is not associated with any consensus instance, and therefore there is no need for //applications to log it or keep any proof. - if (executor instanceof FIFOExecutable) { - response = ((FIFOExecutable) executor).executeUnorderedFIFO(message.getContent(), msgCtx, message.getSender(), message.getOperationId()); - } else { - response = executor.executeUnordered(message.getContent(), msgCtx); - } - - if (message.getReqType() == TOMMessageType.UNORDERED_HASHED_REQUEST - && message.getReplyServer() != this.id) { - response = TOMUtil.computeHash(response); - } - - // Generate the messages to send back to the clients - message.reply = new TOMMessage(id, message.getSession(), message.getSequence(), message.getOperationId(), - response, SVController.getCurrentViewId(), message.getReqType()); - - if (SVController.getStaticConf().getNumRepliers() > 0) { - repMan.send(message); - } else { - cs.send(new int[]{message.getSender()}, message.reply); + response = executor.executeUnordered(id, SVController.getCurrentViewId(), + (message.getReqType() == TOMMessageType.UNORDERED_HASHED_REQUEST && + message.getReplyServer() != this.id),message.getContent(), msgCtx); + + if (response != null) { + if (SVController.getStaticConf().getNumRepliers() > 0) { + repMan.send(response); + } else { + cs.send(new int[]{response.getSender()}, response.reply); + } } } + /** + * Stops the service execution at a replica. It will shutdown all threads, stop the requests' timer, and drop all enqueued requests, + * thus letting the ServiceReplica object be garbage-collected. From the perspective of the rest of the system, this is equivalent + * to a simple crash fault. + */ public void kill() { Thread t = new Thread() { @@ -243,6 +236,10 @@ public void run() { t.start(); } + /** + * Cleans the object state and reboots execution. From the perspective of the rest of the system, + * this is equivalent to a rash followed by a recovery. + */ public void restart() { Thread t = new Thread() { @@ -275,6 +272,10 @@ public void run() { t.start(); } + /** + * + * @deprecated + */ public void receiveMessages(int consId[], int regencies[], int leaders[], CertifiedDecision[] cDecs, TOMMessage[][] requests) { int numRequests = 0; int consensusCount = 0; @@ -321,25 +322,6 @@ public void receiveMessages(int consId[], int regencies[], int leaders[], Certif // deliver requests and contexts to the executor later msgCtxts.add(msgCtx); toBatch.add(request); - } else if (executor instanceof FIFOExecutable) { - - logger.debug("Delivering request from " + request.getSender() + " via FifoExecutable"); - - // This is used to deliver the content decided by a consensus instance directly to - // a Recoverable object. It is useful to allow the application to create a log and - // store the proof associated with decisions (which are needed by replicas - // that are asking for a state transfer). - if (this.recoverer != null) this.recoverer.Op(msgCtx.getConsensusId(), request.getContent(), msgCtx); - - // This is used to deliver the requests to the application and obtain a reply to deliver - //to the clients. The raw decision is passed to the application in the line above. - byte[] response = ((FIFOExecutable) executor).executeOrderedFIFO(request.getContent(), msgCtx, request.getSender(), request.getOperationId()); - - // Generate the messages to send back to the clients - request.reply = new TOMMessage(id, request.getSession(), - request.getSequence(), request.getOperationId(), response, SVController.getCurrentViewId(), request.getReqType()); - logger.debug("sending reply to " + request.getSender()); - replier.manageReply(request, msgCtx); } else if (executor instanceof SingleExecutable) { logger.debug("Delivering request from " + request.getSender() + " via SingleExecutable"); @@ -352,13 +334,13 @@ public void receiveMessages(int consId[], int regencies[], int leaders[], Certif // This is used to deliver the requests to the application and obtain a reply to deliver //to the clients. The raw decision is passed to the application in the line above. - byte[] response = ((SingleExecutable) executor).executeOrdered(request.getContent(), msgCtx); + TOMMessage response = ((SingleExecutable) executor).executeOrdered(id, SVController.getCurrentViewId(), request.getContent(), msgCtx); - // Generate the messages to send back to the clients - request.reply = new TOMMessage(id, request.getSession(), - request.getSequence(), request.getOperationId(), response, SVController.getCurrentViewId(), request.getReqType()); - logger.debug("sending reply to " + request.getSender()); - replier.manageReply(request, msgCtx); + if (response != null) { + + logger.debug("sending reply to " + response.getSender()); + replier.manageReply(response, msgCtx); + } } else { //this code should never be executed throw new UnsupportedOperationException("Non-existent interface"); } break; @@ -440,21 +422,21 @@ public void receiveMessages(int consId[], int regencies[], int leaders[], Certif msgContexts = msgCtxts.toArray(msgContexts); //Deliver the batch and wait for replies - byte[][] replies = ((BatchExecutable) executor).executeBatch(batch, msgContexts); + TOMMessage[] replies = ((BatchExecutable) executor).executeBatch(id, SVController.getCurrentViewId(), batch, msgContexts); //Send the replies back to the client - for (int index = 0; index < toBatch.size(); index++) { - TOMMessage request = toBatch.get(index); - request.reply = new TOMMessage(id, request.getSession(), request.getSequence(), request.getOperationId(), - replies[index], SVController.getCurrentViewId(), request.getReqType()); - - if (SVController.getStaticConf().getNumRepliers() > 0) { - logger.debug("Sending reply to " + request.getSender() + " with sequence number " + request.getSequence() + " and operation ID " + request.getOperationId() +" via ReplyManager"); - repMan.send(request); - } else { - logger.debug("Sending reply to " + request.getSender() + " with sequence number " + request.getSequence() + " and operation ID " + request.getOperationId()); - replier.manageReply(request, msgContexts[index]); - //cs.send(new int[]{request.getSender()}, request.reply); + if (replies != null) { + + for (TOMMessage reply : replies) { + + if (SVController.getStaticConf().getNumRepliers() > 0) { + logger.debug("Sending reply to " + reply.getSender() + " with sequence number " + reply.getSequence() + " and operation ID " + reply.getOperationId() +" via ReplyManager"); + repMan.send(reply); + } else { + logger.debug("Sending reply to " + reply.getSender() + " with sequence number " + reply.getSequence() + " and operation ID " + reply.getOperationId()); + replier.manageReply(reply, null); + //cs.send(new int[]{request.getSender()}, request.reply); + } } } //DEBUG @@ -503,10 +485,11 @@ private void initTOMLayer() { if (SVController.getStaticConf().isShutdownHookEnabled()) { Runtime.getRuntime().addShutdownHook(new ShutdownHookThread(tomLayer)); } + replicaCtx = new ReplicaContext(cs, SVController); + tomLayer.start(); // start the layer execution tomStackCreated = true; - replicaCtx = new ReplicaContext(cs, SVController); } /** @@ -519,11 +502,21 @@ public final ReplicaContext getReplicaContext() { return replicaCtx; } + + /** + * Obtains the current replica communication system. + * + * @return The replica's communication system + */ public ServerCommunicationSystem getServerCommunicationSystem() { return cs; } + /** + * Replica ID + * @return Replica ID + */ public int getId() { return id; } diff --git a/src/bftsmart/tom/core/DeliveryThread.java b/src/bftsmart/tom/core/DeliveryThread.java index 17a8d3850..dd1837e6e 100644 --- a/src/bftsmart/tom/core/DeliveryThread.java +++ b/src/bftsmart/tom/core/DeliveryThread.java @@ -37,297 +37,328 @@ import org.slf4j.LoggerFactory; /** - * This class implements a thread which will deliver totally ordered requests to the application + * This class implements a thread which will deliver totally ordered requests to + * the application * */ public final class DeliveryThread extends Thread { - private Logger logger = LoggerFactory.getLogger(this.getClass()); - - private boolean doWork = true; - private final LinkedBlockingQueue decided; - private final TOMLayer tomLayer; // TOM layer - private final ServiceReplica receiver; // Object that receives requests from clients - private final Recoverable recoverer; // Object that uses state transfer - private final ServerViewController controller; - private final Lock decidedLock = new ReentrantLock(); - private final Condition notEmptyQueue = decidedLock.newCondition(); - - /** - * Creates a new instance of DeliveryThread - * @param tomLayer TOM layer - * @param receiver Object that receives requests from clients - */ - public DeliveryThread(TOMLayer tomLayer, ServiceReplica receiver, Recoverable recoverer, ServerViewController controller) { - super("Delivery Thread"); - this.decided = new LinkedBlockingQueue<>(); - - this.tomLayer = tomLayer; - this.receiver = receiver; - this.recoverer = recoverer; - //******* EDUARDO BEGIN **************// - this.controller = controller; - //******* EDUARDO END **************// - } - - - public Recoverable getRecoverer() { - return recoverer; - } - - /** - * Invoked by the TOM layer, to deliver a decision - * @param dec Decision established from the consensus - */ - public void delivery(Decision dec) { - if (!containsGoodReconfig(dec)) { - - logger.debug("Decision from consensus " + dec.getConsensusId() + " does not contain good reconfiguration"); - //set this decision as the last one from this replica - tomLayer.setLastExec(dec.getConsensusId()); - //define that end of this execution - tomLayer.setInExec(-1); - } //else if (tomLayer.controller.getStaticConf().getProcessId() == 0) System.exit(0); - try { - decidedLock.lock(); - decided.put(dec); - - // clean the ordered messages from the pending buffer - TOMMessage[] requests = extractMessagesFromDecision(dec); - tomLayer.clientsManager.requestsOrdered(requests); - - notEmptyQueue.signalAll(); - decidedLock.unlock(); - logger.debug("Consensus " + dec.getConsensusId() + " finished. Decided size=" + decided.size()); - } catch (Exception e) { - logger.error("Could not insert decision into decided queue",e); - } - } - - private boolean containsGoodReconfig(Decision dec) { - TOMMessage[] decidedMessages = dec.getDeserializedValue(); - - for (TOMMessage decidedMessage : decidedMessages) { - if (decidedMessage.getReqType() == TOMMessageType.RECONFIG - && decidedMessage.getViewID() == controller.getCurrentViewId()) { - return true; - } - } - return false; - } - - /** THIS IS JOAO'S CODE, TO HANDLE STATE TRANSFER */ - private ReentrantLock deliverLock = new ReentrantLock(); - private Condition canDeliver = deliverLock.newCondition(); - - public void deliverLock() { - // release the delivery lock to avoid blocking on state transfer - decidedLock.lock(); - - notEmptyQueue.signalAll(); - decidedLock.unlock(); - - deliverLock.lock(); - } - - public void deliverUnlock() { - deliverLock.unlock(); - } - - public void canDeliver() { - canDeliver.signalAll(); - } - - public void update(ApplicationState state) { - - int lastCID = recoverer.setState(state); - - //set this decision as the last one from this replica - logger.info("Setting last CID to " + lastCID); - tomLayer.setLastExec(lastCID); - - //define the last stable consensus... the stable consensus can - //be removed from the leaderManager and the executionManager - if (lastCID > 2) { - int stableConsensus = lastCID - 3; - tomLayer.execManager.removeOutOfContexts(stableConsensus); - } - - //define that end of this execution - //stateManager.setWaiting(-1); - tomLayer.setNoExec(); - - logger.info("Current decided size: " + decided.size()); - decided.clear(); - - logger.info("All finished up to " + lastCID); - } - - /** - * This is the code for the thread. It delivers decisions to the TOM - * request receiver object (which is the application) - */ - @Override - public void run() { - while (doWork) { - /** THIS IS JOAO'S CODE, TO HANDLE STATE TRANSFER */ - deliverLock(); - while (tomLayer.isRetrievingState()) { - logger.info("Retrieving State"); - canDeliver.awaitUninterruptibly(); - - if (tomLayer.getLastExec() == -1) - logger.info("Ready to process operations"); - } - try { - ArrayList decisions = new ArrayList(); - decidedLock.lock(); - if(decided.isEmpty()) { - notEmptyQueue.await(); - } - - if (controller.getStaticConf().getSameBatchSize()) { - decided.drainTo(decisions, 1); - } else { - decided.drainTo(decisions); - } - - decidedLock.unlock(); - - if (!doWork) break; - - if (decisions.size() > 0) { - TOMMessage[][] requests = new TOMMessage[decisions.size()][]; - int[] consensusIds = new int[requests.length]; - int[] leadersIds = new int[requests.length]; - int[] regenciesIds = new int[requests.length]; - CertifiedDecision[] cDecs; - cDecs = new CertifiedDecision[requests.length]; - int count = 0; - for (Decision d : decisions) { - requests[count] = extractMessagesFromDecision(d); - consensusIds[count] = d.getConsensusId(); - leadersIds[count] = d.getLeader(); - regenciesIds[count] = d.getRegency(); - - CertifiedDecision cDec = new CertifiedDecision(this.controller.getStaticConf().getProcessId(), - d.getConsensusId(), d.getValue(), d.getDecisionEpoch().proof); - cDecs[count] = cDec; - - // cons.firstMessageProposed contains the performance counters - if (requests[count][0].equals(d.firstMessageProposed)) { - long time = requests[count][0].timestamp; - long seed = requests[count][0].seed; - int numOfNonces = requests[count][0].numOfNonces; - requests[count][0] = d.firstMessageProposed; - requests[count][0].timestamp = time; - requests[count][0].seed = seed; - requests[count][0].numOfNonces = numOfNonces; - } - - count++; - } - - Decision lastDecision = decisions.get(decisions.size() - 1); - - if (requests != null && requests.length > 0) { - deliverMessages(consensusIds, regenciesIds, leadersIds, cDecs, requests); - - // ******* EDUARDO BEGIN ***********// - if (controller.hasUpdates()) { - processReconfigMessages(lastDecision.getConsensusId()); - - // set the consensus associated to the last decision as the last executed - tomLayer.setLastExec(lastDecision.getConsensusId()); - // define that end of this execution - tomLayer.setInExec(-1); - // ******* EDUARDO END **************// - } - } - - // define the last stable consensus... the stable consensus can - // be removed from the leaderManager and the executionManager - // TODO: Is this part necessary? If it is, can we put it - // inside setLastExec - int cid = lastDecision.getConsensusId(); - if (cid > 2) { - int stableConsensus = cid - 3; - - tomLayer.execManager.removeConsensus(stableConsensus); - } - } - } catch (Exception e) { - logger.error("Error while processing decision",e); - } - - /** THIS IS JOAO'S CODE, TO HANDLE STATE TRANSFER */ - deliverUnlock(); - /******************************************************************/ - } - logger.info("DeliveryThread stopped."); - - } - - private TOMMessage[] extractMessagesFromDecision(Decision dec) { - TOMMessage[] requests = (TOMMessage[]) dec.getDeserializedValue(); - if (requests == null) { - // there are no cached deserialized requests - // this may happen if this batch proposal was not verified - // TODO: this condition is possible? - - logger.debug("Interpreting and verifying batched requests."); - - // obtain an array of requests from the decisions obtained - BatchReader batchReader = new BatchReader(dec.getValue(), - controller.getStaticConf().getUseSignatures() == 1); - requests = batchReader.deserialiseRequests(controller); - } else { - logger.debug("Using cached requests from the propose."); - } - - return requests; - } - - protected void deliverUnordered(TOMMessage request, int regency) { - - MessageContext msgCtx = new MessageContext(request.getSender(), request.getViewID(), request.getReqType(), - request.getSession(), request.getSequence(), request.getOperationId(), request.getReplyServer(), request.serializedMessageSignature, - System.currentTimeMillis(), 0, 0, regency, -1, -1, null, null, false); // Since the request is unordered, - // there is no consensus info to pass - - msgCtx.readOnly = true; - receiver.receiveReadonlyMessage(request, msgCtx); - } - - private void deliverMessages(int consId[], int regencies[], int leaders[], CertifiedDecision[] cDecs, TOMMessage[][] requests) { - receiver.receiveMessages(consId, regencies, leaders, cDecs, requests); - } - - private void processReconfigMessages(int consId) { - byte[] response = controller.executeUpdates(consId); - TOMMessage[] dests = controller.clearUpdates(); - - if (controller.getCurrentView().isMember(receiver.getId())) { - for (int i = 0; i < dests.length; i++) { - tomLayer.getCommunication().send(new int[]{dests[i].getSender()}, - new TOMMessage(controller.getStaticConf().getProcessId(), - dests[i].getSession(), dests[i].getSequence(), dests[i].getOperationId(), response, - controller.getCurrentViewId(),TOMMessageType.RECONFIG)); - } - - tomLayer.getCommunication().updateServersConnections(); - } else { - receiver.restart(); - } - } - - public void shutdown() { - this.doWork = false; - - logger.info("Shutting down delivery thread"); - - decidedLock.lock(); - notEmptyQueue.signalAll(); - decidedLock.unlock(); - } + private Logger logger = LoggerFactory.getLogger(this.getClass()); + + private boolean doWork = true; + private int lastReconfig = -2; + private final LinkedBlockingQueue decided; + private final TOMLayer tomLayer; // TOM layer + private final ServiceReplica receiver; // Object that receives requests from clients + private final Recoverable recoverer; // Object that uses state transfer + private final ServerViewController controller; + private final Lock decidedLock = new ReentrantLock(); + private final Condition notEmptyQueue = decidedLock.newCondition(); + + /** + * Creates a new instance of DeliveryThread + * + * @param tomLayer TOM layer + * @param receiver Object that receives requests from clients + */ + public DeliveryThread(TOMLayer tomLayer, ServiceReplica receiver, Recoverable recoverer, + ServerViewController controller) { + super("Delivery Thread"); + this.decided = new LinkedBlockingQueue<>(); + + this.tomLayer = tomLayer; + this.receiver = receiver; + this.recoverer = recoverer; + // ******* EDUARDO BEGIN **************// + this.controller = controller; + // ******* EDUARDO END **************// + } + + public Recoverable getRecoverer() { + return recoverer; + } + + /** + * Invoked by the TOM layer, to deliver a decision + * + * @param dec Decision established from the consensus + */ + public void delivery(Decision dec) { + + try { + decidedLock.lock(); + decided.put(dec); + + // clean the ordered messages from the pending buffer + TOMMessage[] requests = extractMessagesFromDecision(dec); + tomLayer.clientsManager.requestsOrdered(requests); + + notEmptyQueue.signalAll(); + decidedLock.unlock(); + logger.debug("Consensus " + dec.getConsensusId() + " finished. Decided size=" + decided.size()); + } catch (Exception e) { + logger.error("Could not insert decision into decided queue", e); + } + + if (!containsReconfig(dec)) { + + logger.debug("Decision from consensus " + dec.getConsensusId() + " does not contain reconfiguration"); + // set this decision as the last one from this replica + tomLayer.setLastExec(dec.getConsensusId()); + // define that end of this execution + tomLayer.setInExec(-1); + } // else if (tomLayer.controller.getStaticConf().getProcessId() == 0) + // System.exit(0); + else { + logger.debug("Decision from consensus " + dec.getConsensusId() + " has reconfiguration"); + lastReconfig = dec.getConsensusId(); + } + } + + private boolean containsReconfig(Decision dec) { + TOMMessage[] decidedMessages = dec.getDeserializedValue(); + + for (TOMMessage decidedMessage : decidedMessages) { + if (decidedMessage.getReqType() == TOMMessageType.RECONFIG + && decidedMessage.getViewID() == controller.getCurrentViewId()) { + return true; + } + } + return false; + } + + /** THIS IS JOAO'S CODE, TO HANDLE STATE TRANSFER */ + private ReentrantLock deliverLock = new ReentrantLock(); + private Condition canDeliver = deliverLock.newCondition(); + + public void deliverLock() { + // release the delivery lock to avoid blocking on state transfer + decidedLock.lock(); + + notEmptyQueue.signalAll(); + decidedLock.unlock(); + + deliverLock.lock(); + } + + public void deliverUnlock() { + deliverLock.unlock(); + } + + public void canDeliver() { + canDeliver.signalAll(); + } + + public void update(ApplicationState state) { + + int lastCID = recoverer.setState(state); + + // set this decision as the last one from this replica + logger.info("Setting last CID to " + lastCID); + tomLayer.setLastExec(lastCID); + + // define the last stable consensus... the stable consensus can + // be removed from the leaderManager and the executionManager + if (lastCID > 2) { + int stableConsensus = lastCID - 3; + tomLayer.execManager.removeOutOfContexts(stableConsensus); + } + + // define that end of this execution + // stateManager.setWaiting(-1); + tomLayer.setNoExec(); + + logger.info("Current decided size: " + decided.size()); + decided.clear(); + + logger.info("All finished up to " + lastCID); + } + + /** + * This is the code for the thread. It delivers decisions to the TOM request + * receiver object (which is the application) + */ + @Override + public void run() { + boolean init = true; + while (doWork) { + /** THIS IS JOAO'S CODE, TO HANDLE STATE TRANSFER */ + deliverLock(); + while (tomLayer.isRetrievingState()) { + logger.info("Retrieving State"); + canDeliver.awaitUninterruptibly(); + + // if (tomLayer.getLastExec() == -1) + if (init) { + logger.info( + "\n\t\t###################################" + + "\n\t\t Ready to process operations " + + "\n\t\t###################################"); + init = false; + } + } + try { + ArrayList decisions = new ArrayList(); + decidedLock.lock(); + if (decided.isEmpty()) { + notEmptyQueue.await(); + } + + logger.debug("Current size of the decided queue: {}", decided.size()); + + if (controller.getStaticConf().getSameBatchSize()) { + decided.drainTo(decisions, 1); + } else { + decided.drainTo(decisions); + } + + decidedLock.unlock(); + + if (!doWork) + break; + + if (decisions.size() > 0) { + TOMMessage[][] requests = new TOMMessage[decisions.size()][]; + int[] consensusIds = new int[requests.length]; + int[] leadersIds = new int[requests.length]; + int[] regenciesIds = new int[requests.length]; + CertifiedDecision[] cDecs; + cDecs = new CertifiedDecision[requests.length]; + int count = 0; + for (Decision d : decisions) { + requests[count] = extractMessagesFromDecision(d); + consensusIds[count] = d.getConsensusId(); + leadersIds[count] = d.getLeader(); + regenciesIds[count] = d.getRegency(); + + CertifiedDecision cDec = new CertifiedDecision(this.controller.getStaticConf().getProcessId(), + d.getConsensusId(), d.getValue(), d.getDecisionEpoch().proof); + cDecs[count] = cDec; + + // cons.firstMessageProposed contains the performance counters + if (requests[count][0].equals(d.firstMessageProposed)) { + long time = requests[count][0].timestamp; + long seed = requests[count][0].seed; + int numOfNonces = requests[count][0].numOfNonces; + requests[count][0] = d.firstMessageProposed; + requests[count][0].timestamp = time; + requests[count][0].seed = seed; + requests[count][0].numOfNonces = numOfNonces; + } + + count++; + } + + Decision lastDecision = decisions.get(decisions.size() - 1); + + if (requests != null && requests.length > 0) { + deliverMessages(consensusIds, regenciesIds, leadersIds, cDecs, requests); + + // ******* EDUARDO BEGIN ***********// + if (controller.hasUpdates()) { + processReconfigMessages(lastDecision.getConsensusId()); + } + if (lastReconfig > -2 && lastReconfig <= lastDecision.getConsensusId()) { + + // set the consensus associated to the last decision as the last executed + logger.debug("Setting last executed consensus to " + lastDecision.getConsensusId()); + tomLayer.setLastExec(lastDecision.getConsensusId()); + // define that end of this execution + tomLayer.setInExec(-1); + // ******* EDUARDO END **************// + + lastReconfig = -2; + } + } + + // define the last stable consensus... the stable consensus can + // be removed from the leaderManager and the executionManager + // TODO: Is this part necessary? If it is, can we put it + // inside setLastExec + int cid = lastDecision.getConsensusId(); + if (cid > 2) { + int stableConsensus = cid - 3; + + tomLayer.execManager.removeConsensus(stableConsensus); + } + } + } catch (Exception e) { + logger.error("Error while processing decision", e); + } + + /** THIS IS JOAO'S CODE, TO HANDLE STATE TRANSFER */ + deliverUnlock(); + /******************************************************************/ + } + logger.info("DeliveryThread stopped."); + + } + + private TOMMessage[] extractMessagesFromDecision(Decision dec) { + TOMMessage[] requests = (TOMMessage[]) dec.getDeserializedValue(); + if (requests == null) { + // there are no cached deserialized requests + // this may happen if this batch proposal was not verified + // TODO: this condition is possible? + + logger.debug("Interpreting and verifying batched requests."); + + // obtain an array of requests from the decisions obtained + BatchReader batchReader = new BatchReader(dec.getValue(), controller.getStaticConf().getUseSignatures()); + requests = batchReader.deserialiseRequests(controller); + } else { + logger.debug("Using cached requests from the propose."); + } + + return requests; + } + + protected void deliverUnordered(TOMMessage request, int regency) { + + MessageContext msgCtx = new MessageContext(request.getSender(), request.getViewID(), request.getReqType(), + request.getSession(), request.getSequence(), request.getOperationId(), request.getReplyServer(), + request.serializedMessageSignature, System.currentTimeMillis(), 0, 0, regency, -1, -1, null, null, + false); // Since the request is unordered, + // there is no consensus info to pass + + msgCtx.readOnly = true; + receiver.receiveReadonlyMessage(request, msgCtx); + } + + private void deliverMessages(int consId[], int regencies[], int leaders[], CertifiedDecision[] cDecs, + TOMMessage[][] requests) { + receiver.receiveMessages(consId, regencies, leaders, cDecs, requests); + } + + private void processReconfigMessages(int consId) { + byte[] response = controller.executeUpdates(consId); + TOMMessage[] dests = controller.clearUpdates(); + + if (controller.getCurrentView().isMember(receiver.getId())) { + for (int i = 0; i < dests.length; i++) { + tomLayer.getCommunication().send(new int[] { dests[i].getSender() }, + new TOMMessage(controller.getStaticConf().getProcessId(), dests[i].getSession(), + dests[i].getSequence(), dests[i].getOperationId(), response, + controller.getCurrentViewId(), TOMMessageType.RECONFIG)); + } + + tomLayer.getCommunication().updateServersConnections(); + } else { + receiver.restart(); + } + } + + public void shutdown() { + this.doWork = false; + + logger.info("Shutting down delivery thread"); + + decidedLock.lock(); + notEmptyQueue.signalAll(); + decidedLock.unlock(); + } + + /* + * public int size() { return decided.size(); } + */ } diff --git a/src/bftsmart/tom/core/ReplyManager.java b/src/bftsmart/tom/core/ReplyManager.java index 9770ac120..4c3fedd6e 100644 --- a/src/bftsmart/tom/core/ReplyManager.java +++ b/src/bftsmart/tom/core/ReplyManager.java @@ -6,9 +6,17 @@ import bftsmart.communication.ServerCommunicationSystem; import bftsmart.tom.core.messages.TOMMessage; +import io.netty.channel.Channel; +import java.util.HashMap; import java.util.LinkedList; +import java.util.Map; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.TimeUnit; +import java.util.concurrent.locks.Condition; +import java.util.concurrent.locks.Lock; +import java.util.concurrent.locks.ReentrantLock; +import java.util.logging.Level; +import java.util.logging.Logger; import org.slf4j.LoggerFactory; @@ -17,7 +25,7 @@ * @author joao */ public class ReplyManager { - + private LinkedList threads; private int iteration; @@ -35,41 +43,58 @@ public ReplyManager(int numThreads, ServerCommunicationSystem cs) { } public void send (TOMMessage msg) { - + iteration++; threads.get((iteration % threads.size())).send(msg); } } class ReplyThread extends Thread { - - private static final long POOL_TIME = 5000; - + private LinkedBlockingQueue replies; private ServerCommunicationSystem cs = null; + private final Lock queueLock = new ReentrantLock(); + private final Condition notEmptyQueue = queueLock.newCondition(); + + private Map channels; + ReplyThread(ServerCommunicationSystem cs) { this.cs = cs; this.replies = new LinkedBlockingQueue(); + this.channels = new HashMap<>(); } void send(TOMMessage msg) { - replies.add(msg); + + try { + queueLock.lock(); + replies.put(msg); + notEmptyQueue.signalAll(); + queueLock.unlock(); + } catch (InterruptedException ex) { + Logger.getLogger(ReplyThread.class.getName()).log(Level.SEVERE, null, ex); + } } public void run() { - TOMMessage msg; while (true) { try { - msg = replies.poll(POOL_TIME, TimeUnit.MILLISECONDS); - if (msg == null) { - - continue; //go back to the start of the loop + + LinkedList list = new LinkedList<>(); + + queueLock.lock(); + while (replies.isEmpty()) notEmptyQueue.await(10, TimeUnit.MILLISECONDS); + replies.drainTo(list); + queueLock.unlock(); + + for (TOMMessage msg : list) { + + cs.getClientsConn().send(new int[] {msg.getSender()}, msg.reply, false); } - cs.getClientsConn().send(new int[] {msg.getSender()}, msg.reply, false); } catch (InterruptedException ex) { LoggerFactory.getLogger(this.getClass()).error("Could not retrieve reply from queue",ex); } diff --git a/src/bftsmart/tom/core/Synchronizer.java b/src/bftsmart/tom/core/Synchronizer.java index ca3cbe017..13cd17857 100644 --- a/src/bftsmart/tom/core/Synchronizer.java +++ b/src/bftsmart/tom/core/Synchronizer.java @@ -150,7 +150,7 @@ public void triggerTimeout(List requestList) { //TODO: If this is null, then there was no timeout nor STOP messages. //What to do? - byte[] serialized = bb.makeBatch(messages, 0, 0, controller); + byte[] serialized = bb.makeBatch(messages, 0, 0, controller.getStaticConf().getUseSignatures()); out.writeBoolean(true); out.writeObject(serialized); } else { @@ -361,7 +361,7 @@ private TOMMessage[] deserializeTOMMessages(byte[] playload) { //TODO: The requests have to be verified! byte[] temp = (byte[]) ois.readObject(); BatchReader batchReader = new BatchReader(temp, - controller.getStaticConf().getUseSignatures() == 1); + controller.getStaticConf().getUseSignatures()); requests = batchReader.deserialiseRequests(controller); } else { @@ -481,7 +481,7 @@ private void startSynchronization(int nextReg) { //TODO: If this is null, there was no timeout nor STOP messages. //What shall be done then? out.writeBoolean(true); - byte[] serialized = bb.makeBatch(messages, 0, 0, controller); + byte[] serialized = bb.makeBatch(messages, 0, 0, controller.getStaticConf().getUseSignatures()); out.writeObject(serialized); } else { out.writeBoolean(false); @@ -1201,10 +1201,12 @@ else if (cm.getType() == MessageFactory.WRITE) { logger.info("Sending WRITE message for CID " + currentCID + ", timestamp " + e.getTimestamp() + ", value " + Arrays.toString(e.propValueHash)); communication.send(this.controller.getCurrentViewOtherAcceptors(), acceptor.getFactory().createWrite(currentCID, e.getTimestamp(), e.propValueHash)); + e.writeSent(); } else { logger.info("Sending ACCEPT message for CID " + currentCID + ", timestamp " + e.getTimestamp() + ", value " + Arrays.toString(e.propValueHash)); communication.send(this.controller.getCurrentViewOtherAcceptors(), acceptor.getFactory().createAccept(currentCID, e.getTimestamp(), e.propValueHash)); + e.acceptSent(); } } else { logger.warn("Sync phase failed for regency" + regency); diff --git a/src/bftsmart/tom/core/TOMLayer.java b/src/bftsmart/tom/core/TOMLayer.java index 4529eec5d..58f2d6048 100644 --- a/src/bftsmart/tom/core/TOMLayer.java +++ b/src/bftsmart/tom/core/TOMLayer.java @@ -19,6 +19,7 @@ import java.io.Serializable; import java.security.MessageDigest; import java.security.PrivateKey; +import java.security.PublicKey; import java.security.Signature; import java.security.SignedObject; import java.util.concurrent.locks.Condition; @@ -44,8 +45,15 @@ import bftsmart.tom.util.BatchBuilder; import bftsmart.tom.util.BatchReader; import bftsmart.tom.util.TOMUtil; + +import java.util.HashMap; +import java.util.Timer; +import java.util.TimerTask; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; /** * This class implements the state machine replication protocol described in @@ -66,10 +74,18 @@ public final class TOMLayer extends Thread implements RequestReceiver { private DeliveryThread dt; // Thread which delivers total ordered messages to the appication public StateManager stateManager = null; // object which deals with the state transfer protocol + //thread pool used to paralelise verification of requests contained in a batch + private ExecutorService verifierExecutor = null; + /** * Manage timers for pending requests */ public RequestsTimer requestsTimer; + + //timeout for batch + private Timer batchTimer = null; + private long lastRequest = -1; + /** * Store requests received but still not ordered */ @@ -96,7 +112,9 @@ public final class TOMLayer extends Thread implements RequestReceiver { private ReentrantLock proposeLock = new ReentrantLock(); private Condition canPropose = proposeLock.newCondition(); - private PrivateKey prk; + private PrivateKey privateKey; + private HashMap publicKey; + public ServerViewController controller; private RequestVerifier verifier; @@ -128,14 +146,27 @@ public TOMLayer(ExecutionManager manager, this.acceptor = a; this.communication = cs; this.controller = controller; - + + /*Tulio Ribeiro*/ + this.privateKey = this.controller.getStaticConf().getPrivateKey(); + this.publicKey = new HashMap<>(); + int [] targets = this.controller.getCurrentViewAcceptors(); + for (int i = 0; i < targets.length; i++) { + publicKey.put(targets[i], controller.getStaticConf().getPublicKey(targets[i])); + } + // use either the same number of Netty workers threads if specified in the configuration + // or use a many as the number of cores available + int nWorkers = this.controller.getStaticConf().getNumNettyWorkers(); + nWorkers = nWorkers > 0 ? nWorkers : Runtime.getRuntime().availableProcessors(); + this.verifierExecutor = Executors.newWorkStealingPool(nWorkers); + //do not create a timer manager if the timeout is 0 if (this.controller.getStaticConf().getRequestTimeout() == 0) { this.requestsTimer = null; } else { this.requestsTimer = new RequestsTimer(this, communication, this.controller); // Create requests timers manager (a thread) } - + try { this.md = TOMUtil.getHashEngine(); } catch (Exception e) { @@ -148,7 +179,6 @@ public TOMLayer(ExecutionManager manager, logger.error("Failed to get signature engine",e); } - this.prk = this.controller.getStaticConf().getPrivateKey(); this.dt = new DeliveryThread(this, receiver, recoverer, this.controller); // Create delivery thread this.dt.start(); this.stateManager = recoverer.getStateManager(); @@ -160,6 +190,24 @@ public TOMLayer(ExecutionManager manager, this.clientsManager = new ClientsManager(this.controller, requestsTimer, this.verifier); this.syncher = new Synchronizer(this); // create synchronizer + + if (controller.getStaticConf().getBatchTimeout() > -1) { + + batchTimer = new Timer(); + batchTimer.scheduleAtFixedRate(new TimerTask() { + @Override + public void run() { + + if (clientsManager.havePendingRequests() && + (System.currentTimeMillis() - lastRequest) >= controller.getStaticConf().getBatchTimeout()) { + + logger.debug("Signaling proposer thread!!"); + haveMessages(); + } + } + + }, 0, controller.getStaticConf().getBatchTimeout()); + } } /** @@ -179,7 +227,7 @@ public final byte[] computeHash(byte[] data) { public SignedObject sign(Serializable obj) { try { - return new SignedObject(obj, prk, engine); + return new SignedObject(obj, privateKey, engine); } catch (Exception e) { logger.error("Failed to sign object",e); return null; @@ -195,7 +243,7 @@ public SignedObject sign(Serializable obj) { */ public boolean verifySignature(SignedObject so, int sender) { try { - return so.verify(controller.getStaticConf().getPublicKey(sender), engine); + return so.verify(publicKey.get(sender), engine); } catch (Exception e) { logger.error("Failed to verify object signature",e); } @@ -278,7 +326,7 @@ public int getInExec() { */ @Override public void requestReceived(TOMMessage msg) { - + if (!doWork) return; // check if this request is valid and add it to the client' pending requests list @@ -292,7 +340,21 @@ public void requestReceived(TOMMessage msg) { logger.debug("Received TOMMessage from client " + msg.getSender() + " with sequence number " + msg.getSequence() + " for session " + msg.getSession()); if (clientsManager.requestReceived(msg, true, communication)) { - haveMessages(); + + if(controller.getStaticConf().getBatchTimeout() == -1) { + haveMessages(); + } else { + + if (clientsManager.countPendingRequests() < controller.getStaticConf().getMaxBatchSize()) { + + lastRequest = System.currentTimeMillis(); + + } else { + + haveMessages(); + } + + } } else { logger.warn("The received TOMMessage " + msg + " was discarded."); } @@ -309,6 +371,8 @@ public void requestReceived(TOMMessage msg) { public byte[] createPropose(Decision dec) { // Retrieve a set of pending requests from the clients manager RequestList pendingRequests = clientsManager.getPendingRequests(); + + logger.debug("Number of pending requets to propose in consensus {}: {}", dec.getConsensusId(), pendingRequests.size()); int numberOfMessages = pendingRequests.size(); // number of messages retrieved int numberOfNonces = this.controller.getStaticConf().getNumberOfNonces(); // ammount of nonces to be generated @@ -322,7 +386,7 @@ public byte[] createPropose(Decision dec) { logger.debug("Creating a PROPOSE with " + numberOfMessages + " msgs"); - return bb.makeBatch(pendingRequests, numberOfNonces, System.currentTimeMillis(), controller); + return bb.makeBatch(pendingRequests, numberOfNonces, System.currentTimeMillis(), controller.getStaticConf().getUseSignatures()); } /** @@ -364,16 +428,20 @@ public void run() { // blocks until there are requests to be processed/ordered messagesLock.lock(); - if (!clientsManager.havePendingRequests()) { + if (!clientsManager.havePendingRequests() || + (controller.getStaticConf().getBatchTimeout() > -1 + && clientsManager.countPendingRequests() < controller.getStaticConf().getMaxBatchSize())) { + + logger.debug("Waiting for enough requests"); haveMessages.awaitUninterruptibly(); + logger.debug("Got enough requests"); } messagesLock.unlock(); if (!doWork) break; - logger.debug("There are messages to be ordered."); + logger.debug("There are requests to be ordered. I will propose."); - logger.debug("I can try to propose."); if ((execManager.getCurrentLeader() == this.controller.getStaticConf().getProcessId()) && //I'm the leader (clientsManager.havePendingRequests()) && //there are messages to be ordered @@ -389,7 +457,7 @@ public void run() { if (controller.getCurrentViewN() == 1) { logger.debug("Only one replica, bypassing consensus."); - + byte[] value = createPropose(dec); Consensus consensus = execManager.getConsensus(dec.getConsensusId()); @@ -406,8 +474,7 @@ public void run() { continue; } - execManager.getProposer().startConsensus(execId, - createPropose(dec)); + execManager.getProposer().startConsensus(execId, createPropose(dec)); } } logger.info("TOMLayer stopped."); @@ -434,16 +501,17 @@ public void decided(Decision dec) { * TODO: verify timestamps and nonces * * @param proposedValue the value being proposed + * @param addToClientManager add the requests to the client manager * @return Valid messages contained in the proposed value */ public TOMMessage[] checkProposedValue(byte[] proposedValue, boolean addToClientManager) { - + try{ logger.debug("Checking proposed value"); BatchReader batchReader = new BatchReader(proposedValue, - this.controller.getStaticConf().getUseSignatures() == 1); + this.controller.getStaticConf().getUseSignatures()); TOMMessage[] requests = null; @@ -451,21 +519,40 @@ public TOMMessage[] checkProposedValue(byte[] proposedValue, boolean addToClient //TODO: verify Timestamps and Nonces requests = batchReader.deserialiseRequests(this.controller); - //enforce the "external validity" property, i.e, verify if the - //requests are valid in accordance to the application semantics - //and not an erroneous requests sent by a Byzantine leader. - for (TOMMessage r : requests) { - if (controller.getStaticConf().isBFT() &&!verifier.isValidRequest(r)) return null; - } - - if (addToClientManager) { - for (int i = 0; i < requests.length; i++) { - //notifies the client manager that this request was received and get - //the result of its validation - if (!clientsManager.requestReceived(requests[i], false)) { - clientsManager.getClientsLock().unlock(); - logger.warn("Request could not be added to the pending messages queue of its respective client"); + + //use parallelization to validate the request + final CountDownLatch latch = new CountDownLatch(requests.length); + + for (TOMMessage request : requests) { + + verifierExecutor.submit(() -> { + try { + + //notifies the client manager that this request was received and get + //the result of its validation + request.isValid = clientsManager.requestReceived(request, false); + if (Thread.holdsLock(clientsManager.getClientsLock())) clientsManager.getClientsLock().unlock(); + + } + catch (Exception e) { + + logger.error("Error while validating requests", e); + if (Thread.holdsLock(clientsManager.getClientsLock())) clientsManager.getClientsLock().unlock(); + + } + + latch.countDown(); + }); + } + + latch.await(); + + for (TOMMessage request : requests) { + + if (request.isValid == false) { + + logger.warn("Request {} could not be added to the pending messages queue of its respective client", request); return null; } } diff --git a/src/bftsmart/tom/TOMSender.java b/src/bftsmart/tom/core/TOMSender.java similarity index 95% rename from src/bftsmart/tom/TOMSender.java rename to src/bftsmart/tom/core/TOMSender.java index dfae581a3..63952845a 100644 --- a/src/bftsmart/tom/TOMSender.java +++ b/src/bftsmart/tom/core/TOMSender.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.tom; +package bftsmart.tom.core; import java.util.Random; import java.util.concurrent.atomic.AtomicInteger; @@ -75,13 +75,13 @@ public ClientViewController getViewManager(){ * * @param processId ID of the process */ - public void init(int processId, KeyLoader loader, Provider provider) { - this.viewController = new ClientViewController(processId, loader, provider); + public void init(int processId, KeyLoader loader) { + this.viewController = new ClientViewController(processId, loader); startsCS(processId); } - public void init(int processId, String configHome, KeyLoader loader, Provider provider) { - this.viewController = new ClientViewController(processId,configHome, loader, provider); + public void init(int processId, String configHome, KeyLoader loader) { + this.viewController = new ClientViewController(processId,configHome, loader); startsCS(processId); } @@ -89,7 +89,7 @@ private void startsCS(int clientId) { this.cs = CommunicationSystemClientSideFactory.getCommunicationSystemClientSide(clientId, this.viewController); this.cs.setReplyReceiver(this); // This object itself shall be a reply receiver this.me = this.viewController.getStaticConf().getProcessId(); - this.useSignatures = this.viewController.getStaticConf().getUseSignatures()==1?true:false; + this.useSignatures = this.viewController.getStaticConf().getUseSignatures(); this.session = new Random().nextInt(); } //******* EDUARDO END **************// diff --git a/src/bftsmart/tom/core/messages/TOMMessage.java b/src/bftsmart/tom/core/messages/TOMMessage.java index 4974aa476..bb2bb0181 100644 --- a/src/bftsmart/tom/core/messages/TOMMessage.java +++ b/src/bftsmart/tom/core/messages/TOMMessage.java @@ -57,11 +57,14 @@ public class TOMMessage extends SystemMessage implements Externalizable, Compara public transient int destination = -1; // message destination public transient boolean signed = false; // is this message signed? - public transient long receptionTime;//the reception time of this message - public transient boolean timeout = false;//this message was timed out? + public transient long receptionTime;//the reception time of this message (nanoseconds) + public transient long receptionTimestamp;//the reception timestamp of this message (miliseconds) + + public transient boolean timeout = false;//this message was timed out? public transient boolean recvFromClient = false; // Did the client already sent this message to me, or did it arrived in the batch? - + public transient boolean isValid = false; // Was this request already validated by the replica? + //the bytes received from the client and its MAC and signature public transient byte[] serializedMessage = null; public transient byte[] serializedMessageSignature = null; @@ -219,7 +222,7 @@ public int hashCode() { @Override public String toString() { - return "(" + sender + "," + sequence + "," + operationId + "," + session + ")"; + return "[" + sender + ":" + session + ":" + sequence + "]"; } public void wExternal(DataOutput out) throws IOException { @@ -338,8 +341,42 @@ public int compareTo(Object o) { return EQUAL; } + @Override public Object clone() throws CloneNotSupportedException { - return super.clone(); + + + TOMMessage clone = new TOMMessage(sender, session, sequence, + operationId, content, viewID, type); + + clone.setReplyServer(replyServer); + + clone.acceptSentTime = this.acceptSentTime; + clone.alreadyProposed = this.alreadyProposed; + clone.authenticated = this.authenticated; + clone.consensusStartTime = this.consensusStartTime; + clone.decisionTime = this.decisionTime; + clone.deliveryTime = this.deliveryTime; + clone.destination = this.destination; + clone.executedTime = this.executedTime; + clone.info = this.info; + clone.isValid = this.isValid; + clone.numOfNonces = this.numOfNonces; + clone.proposeReceivedTime = this.proposeReceivedTime; + clone.receptionTime = this.receptionTime; + clone.receptionTimestamp = this.receptionTimestamp; + clone.recvFromClient = this.recvFromClient; + clone.reply = this.reply; + clone.seed = this.seed; + clone.serializedMessage = this.serializedMessage; + clone.serializedMessageMAC = this.serializedMessageMAC; + clone.serializedMessageSignature = this.serializedMessageSignature; + clone.signed = this.signed; + clone.timeout = this.timeout; + clone.timestamp = this.timestamp; + clone.writeSentTime = this.writeSentTime; + + return clone; + } diff --git a/src/bftsmart/tom/leaderchange/LCManager.java b/src/bftsmart/tom/leaderchange/LCManager.java index e427fa95a..6df1a236e 100644 --- a/src/bftsmart/tom/leaderchange/LCManager.java +++ b/src/bftsmart/tom/leaderchange/LCManager.java @@ -15,33 +15,28 @@ */ package bftsmart.tom.leaderchange; -import bftsmart.communication.server.ServerConnection; +import java.io.ByteArrayOutputStream; import java.io.IOException; +import java.io.ObjectOutputStream; import java.security.MessageDigest; +import java.security.PublicKey; import java.security.SignedObject; import java.util.Arrays; import java.util.HashMap; import java.util.HashSet; +import java.util.LinkedList; import java.util.List; import java.util.Set; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + import bftsmart.consensus.TimestampValuePair; import bftsmart.consensus.messages.ConsensusMessage; import bftsmart.reconfiguration.ServerViewController; import bftsmart.tom.core.TOMLayer; import bftsmart.tom.core.messages.TOMMessage; import bftsmart.tom.util.TOMUtil; -import java.io.ByteArrayOutputStream; -import java.io.ObjectOutputStream; -import java.security.InvalidKeyException; -import java.security.NoSuchAlgorithmException; -import java.security.PublicKey; -import java.util.LinkedList; -import javax.crypto.Mac; -import javax.crypto.SecretKey; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; /** * @@ -77,7 +72,6 @@ public class LCManager { private int currentLeader; //private Cipher cipher; - private Mac mac; /** * Constructor @@ -98,12 +92,6 @@ public LCManager(TOMLayer tomLayer,ServerViewController SVController, MessageDig this.SVController = SVController; this.md = md; - try { - this.mac = TOMUtil.getMacFactory(); - } catch (NoSuchAlgorithmException /*| NoSuchPaddingException*/ ex) { - logger.error("Could not instantiate MAC algorithm",ex); - } - } /** @@ -481,8 +469,6 @@ public boolean binds(int timestamp, byte[] value, HashSet collects) return (quorumHighest(timestamp, value, collects) && certifiedValue(timestamp, value, collects)); - //return value != null && collects != null && (collects.size() >= (SVController.getCurrentViewN() - SVController.getCurrentViewF())) - // && quorumHighest(timestamp, value, collects) && certifiedValue(timestamp, value, collects); } /** @@ -808,12 +794,10 @@ public boolean hasValidProof(CertifiedDecision cDec) { byte[] hashedValue = md.digest(cDec.getDecision()); Set ConsensusMessages = cDec.getConsMessages(); - int myId = tomLayer.controller.getStaticConf().getProcessId(); int certificateCurrentView = (2*tomLayer.controller.getCurrentViewF()) + 1; int certificateLastView = -1; if (tomLayer.controller.getLastView() != null) certificateLastView = (2*tomLayer.controller.getLastView().getF()) + 1; int countValid = 0; - SecretKey secretKey = null; PublicKey pubKey = null; HashSet alreadyCounted = new HashSet<>(); //stores replica IDs that were already counted @@ -832,46 +816,24 @@ public boolean hasValidProof(CertifiedDecision cDec) { byte[] data = bOut.toByteArray(); - if (consMsg.getProof() instanceof HashMap) { // Certificate is made of MAC vector - - logger.debug("Proof made of MAC vector"); - - HashMap macVector = (HashMap) consMsg.getProof(); - - byte[] recvMAC = macVector.get(myId); - - byte[] myMAC = null; - - secretKey = tomLayer.getCommunication().getServersConn().getSecretKey(consMsg.getSender()); - try { - this.mac.init(secretKey); - myMAC = this.mac.doFinal(data); - } catch (InvalidKeyException ex) { - logger.error("Could not compute MAC",ex); - } - - if (recvMAC != null && myMAC != null && Arrays.equals(recvMAC, myMAC) && - Arrays.equals(consMsg.getValue(), hashedValue) && - consMsg.getNumber() == cDec.getCID() && !alreadyCounted.contains(consMsg.getSender())) { - - alreadyCounted.add(consMsg.getSender()); - countValid++; - } - } else if (consMsg.getProof() instanceof byte[]) { // certificate is made of signatures + if (consMsg.getProof() instanceof byte[]) { // certificate is made of signatures logger.debug("Proof made of Signatures"); pubKey = SVController.getStaticConf().getPublicKey(consMsg.getSender()); byte[] signature = (byte[]) consMsg.getProof(); - if (TOMUtil.verifySignature(pubKey, data, signature) && !alreadyCounted.contains(consMsg.getSender())) { + if (Arrays.equals(consMsg.getValue(), hashedValue) && + TOMUtil.verifySignature(pubKey, data, signature) && !alreadyCounted.contains(consMsg.getSender())) { alreadyCounted.add(consMsg.getSender()); countValid++; + } else { + logger.error("Invalid signature in message from " + consMsg.getSender()); } } else { - logger.debug("Proof is message is invalid"); + logger.debug("Proof message is not an instance of byte[]."); } } @@ -883,7 +845,10 @@ public boolean hasValidProof(CertifiedDecision cDec) { logger.debug("Computing certificate based on previous view"); //return countValid >= certificateCurrentView; - return countValid >= (certificateLastView != -1 && pubKey != null ? certificateLastView : certificateCurrentView); + boolean ret = countValid >= (certificateLastView != -1 && pubKey != null ? certificateLastView : certificateCurrentView); + logger.debug("Proof for CID {} is {} ({} valid messages, needed {})", + cDec.getCID(), (ret ? "valid" : "invalid"), countValid, ( pubKey != null ? certificateLastView : certificateCurrentView)); + return ret; } /** diff --git a/src/bftsmart/tom/leaderchange/RequestsTimer.java b/src/bftsmart/tom/leaderchange/RequestsTimer.java index e0427ef6d..c9c7ed038 100644 --- a/src/bftsmart/tom/leaderchange/RequestsTimer.java +++ b/src/bftsmart/tom/leaderchange/RequestsTimer.java @@ -77,14 +77,6 @@ public void setShortTimeout(long shortTimeout) { this.shortTimeout = shortTimeout; } - public void setTimeout(long timeout) { - this.timeout = timeout; - } - - public long getTimeout() { - return timeout; - } - public void startTimer() { if (rtTask == null) { long t = (shortTimeout > -1 ? shortTimeout : timeout); @@ -157,25 +149,33 @@ public void run_lc_protocol() { //System.out.println("(RequestTimerTask.run) I SOULD NEVER RUN WHEN THERE IS NO TIMEOUT"); - LinkedList pendingRequests = new LinkedList(); + LinkedList pendingRequests = new LinkedList<>(); - rwLock.readLock().lock(); + try { - for (Iterator i = watched.iterator(); i.hasNext();) { - TOMMessage request = i.next(); - if ((request.receptionTime + System.currentTimeMillis()) > t) { - pendingRequests.add(request); - } else { - break; + rwLock.readLock().lock(); + + for (Iterator i = watched.iterator(); i.hasNext();) { + TOMMessage request = i.next(); + if ((System.currentTimeMillis() - request.receptionTimestamp ) > t) { + pendingRequests.add(request); + } } + + } finally { + + rwLock.readLock().unlock(); } - - rwLock.readLock().unlock(); - + if (!pendingRequests.isEmpty()) { + + logger.info("The following requests timed out: " + pendingRequests); + for (ListIterator li = pendingRequests.listIterator(); li.hasNext(); ) { TOMMessage request = li.next(); if (!request.timeout) { + + logger.info("Forwarding requests {} to leader", request); request.signed = request.serializedMessageSignature != null; tomLayer.forwardRequestToLeader(request); @@ -185,7 +185,7 @@ public void run_lc_protocol() { } if (!pendingRequests.isEmpty()) { - logger.info("Timeout for messages: " + pendingRequests); + logger.info("Attempting to start leader change for requests {}", pendingRequests); //Logger.debug = true; //tomLayer.requestTimeout(pendingRequests); //if (reconfManager.getStaticConf().getProcessId() == 4) Logger.debug = true; @@ -196,8 +196,11 @@ public void run_lc_protocol() { timer.schedule(rtTask, t); } } else { - rtTask = null; - timer.purge(); + + logger.debug("Timeout triggered with no expired requests"); + + rtTask = new RequestTimerTask(); + timer.schedule(rtTask, t); } } diff --git a/src/bftsmart/tom/server/BatchExecutable.java b/src/bftsmart/tom/server/BatchExecutable.java index b5cdcda4e..604c849c6 100644 --- a/src/bftsmart/tom/server/BatchExecutable.java +++ b/src/bftsmart/tom/server/BatchExecutable.java @@ -16,23 +16,36 @@ package bftsmart.tom.server; import bftsmart.tom.MessageContext; +import bftsmart.tom.core.messages.TOMMessage; /** * * Executables that implement this interface will receive a batch of requests and * deliver them to the application in a deterministic way. - * - * @author Marcel Santos * */ public interface BatchExecutable extends Executable { - /** - * Execute a batch of requests. - * @param command - * @param msgCtx - * @return - */ + /** + * Execute a batch of requests. + * @param command The batch of requests + * @param msgCtx The context associated to each request + * @return + */ public byte[][] executeBatch(byte[][] command, MessageContext[] msgCtx); + + public default TOMMessage[] executeBatch(int processID, int viewID, byte[][] command, MessageContext[] msgCtx) { + + TOMMessage[] replies = new TOMMessage[command.length]; + + byte[][] results = executeBatch(command, msgCtx); + + for (int i = 0; i < results.length; i++) { + + replies[i] = getTOMMessage(processID, viewID, command[i], msgCtx[i], results[i]); + } + + return replies; + } } diff --git a/src/bftsmart/tom/server/Executable.java b/src/bftsmart/tom/server/Executable.java index db8400b74..4ba9a3d47 100644 --- a/src/bftsmart/tom/server/Executable.java +++ b/src/bftsmart/tom/server/Executable.java @@ -16,14 +16,14 @@ package bftsmart.tom.server; import bftsmart.tom.MessageContext; +import bftsmart.tom.core.messages.TOMMessage; +import bftsmart.tom.util.TOMUtil; /** * * Executables that implement this interface can received unordered client requests. * To support ordered requests, objects that implement this interface must also implement * either 'FIFOExecutable', 'BatchExecutable' or 'SingleExecutable'. - * - * @author Marcel Santos * */ public interface Executable { @@ -40,4 +40,22 @@ public interface Executable { * @return the reply for the request issued by the client */ public byte[] executeUnordered(byte[] command, MessageContext msgCtx); + + default TOMMessage getTOMMessage(int processID, int viewID, byte[] command, MessageContext msgCtx, byte[] result) { + + TOMMessage reply = msgCtx.recreateTOMMessage(command); + reply.reply = new TOMMessage(processID, reply.getSession(), reply.getSequence(), reply.getOperationId(), + result, viewID, reply.getReqType()); + + return reply; + } + + public default TOMMessage executeUnordered(int processID, int viewID, boolean isReplyHash, byte[] command, MessageContext msgCtx) { + + byte[] result = executeUnordered(command, msgCtx); + + if (isReplyHash) result = TOMUtil.computeHash(result); + + return getTOMMessage(processID, viewID, command, msgCtx, result); + } } diff --git a/src/bftsmart/tom/server/FIFOExecutable.java b/src/bftsmart/tom/server/FIFOExecutable.java deleted file mode 100644 index b2ade61db..000000000 --- a/src/bftsmart/tom/server/FIFOExecutable.java +++ /dev/null @@ -1,33 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.tom.server; - -import bftsmart.tom.MessageContext; - -/** - * - * Executables that implement this interface will get requests - * delivered in FIFO order. - * - * @author Marcel Santos - * - */ -public interface FIFOExecutable extends SingleExecutable { - - public byte[] executeOrderedFIFO(byte[] command, MessageContext msgCtx, int clientId, int operationId); - public byte[] executeUnorderedFIFO(byte[] command, MessageContext msgCtx, int clientId, int operationId); - -} diff --git a/src/bftsmart/tom/server/Recoverable.java b/src/bftsmart/tom/server/Recoverable.java index 5bdfec027..56d292fd8 100644 --- a/src/bftsmart/tom/server/Recoverable.java +++ b/src/bftsmart/tom/server/Recoverable.java @@ -23,13 +23,15 @@ /** * Classes that implement this interface should implement a state transfer protocol. * Typically, classes should both implement this interface and one of the executables. - * - * @author Marcel Santos * */ public interface Recoverable { - public void setReplicaContext(ReplicaContext replicaContext); + /** + * Sets the replica context + * @param replicaContext The replica context + */ + public void setReplicaContext(ReplicaContext replicaContext); /** * diff --git a/src/bftsmart/tom/server/Replier.java b/src/bftsmart/tom/server/Replier.java index 2d0e4dea5..ed326a512 100644 --- a/src/bftsmart/tom/server/Replier.java +++ b/src/bftsmart/tom/server/Replier.java @@ -23,12 +23,21 @@ * Provides support for building custom reply management * to be used in the ServiceReplica. * - * @author Miguel Garcia */ public interface Replier { - public void setReplicaContext(ReplicaContext rc); - + /** + * Sets the replica context + * @param replicaContext The replica context + */ + public void setReplicaContext(ReplicaContext replicaContext); + + /** + * Given an executed request, send it to a target + * + * @param request The executed request + * @param msgCtx The message context associated to the request + */ public void manageReply(TOMMessage request, MessageContext msgCtx); } diff --git a/src/bftsmart/tom/server/RequestVerifier.java b/src/bftsmart/tom/server/RequestVerifier.java index 11ba33676..79b5c6374 100644 --- a/src/bftsmart/tom/server/RequestVerifier.java +++ b/src/bftsmart/tom/server/RequestVerifier.java @@ -16,10 +16,15 @@ * are valid in accordance to the application semantics (and not * an erroneous requests sent by a Byzantine leader). * - * @author joao */ public interface RequestVerifier { + /** + * Given a request, validated it + * + * @param request The request to be validated + * @return true if the request is valid, false otherwise + */ public boolean isValidRequest(TOMMessage request); } diff --git a/src/bftsmart/tom/server/SingleExecutable.java b/src/bftsmart/tom/server/SingleExecutable.java index 5bec4ed31..7ea22994e 100644 --- a/src/bftsmart/tom/server/SingleExecutable.java +++ b/src/bftsmart/tom/server/SingleExecutable.java @@ -16,11 +16,10 @@ package bftsmart.tom.server; import bftsmart.tom.MessageContext; +import bftsmart.tom.core.messages.TOMMessage; /** * Executables that implement this interface will receive client requests individually. - * - * @author Marcel Santos * */ public interface SingleExecutable extends Executable { @@ -42,5 +41,13 @@ public interface SingleExecutable extends Executable { * @return the reply for the request issued by the client */ public byte[] executeOrdered(byte[] command, MessageContext msgCtx); + + public default TOMMessage executeOrdered(int processID, int viewID, byte[] command, MessageContext msgCtx) { + + byte[] result = executeOrdered(command, msgCtx); + + return getTOMMessage(processID, viewID, command, msgCtx, result); + + } } diff --git a/src/bftsmart/tom/server/defaultservices/DefaultApplicationState.java b/src/bftsmart/tom/server/defaultservices/DefaultApplicationState.java index 58fc7f555..1ddbab88a 100644 --- a/src/bftsmart/tom/server/defaultservices/DefaultApplicationState.java +++ b/src/bftsmart/tom/server/defaultservices/DefaultApplicationState.java @@ -137,7 +137,7 @@ public CertifiedDecision getCertifiedDecision(ServerViewController controller) { //Serialize the TOMMessages to re-create the proposed value BatchBuilder bb = new BatchBuilder(0); byte[] value = bb.makeBatch(requests, ci.msgCtx[0].getNumOfNonces(), - ci.msgCtx[0].getSeed(), ci.msgCtx[0].getTimestamp(), controller); + ci.msgCtx[0].getSeed(), ci.msgCtx[0].getTimestamp(), controller.getStaticConf().getUseSignatures()); //Assemble and return the certified decision return new CertifiedDecision(pid, getLastCID(), value, proof); diff --git a/src/bftsmart/tom/server/defaultservices/DefaultRecoverable.java b/src/bftsmart/tom/server/defaultservices/DefaultRecoverable.java index d58b2cef2..f8848bc6e 100644 --- a/src/bftsmart/tom/server/defaultservices/DefaultRecoverable.java +++ b/src/bftsmart/tom/server/defaultservices/DefaultRecoverable.java @@ -27,7 +27,7 @@ import bftsmart.reconfiguration.util.TOMConfiguration; import bftsmart.statemanagement.ApplicationState; import bftsmart.statemanagement.StateManager; -import bftsmart.statemanagement.strategy.StandardStateManager; +import bftsmart.statemanagement.standard.StandardStateManager; import bftsmart.tom.MessageContext; import bftsmart.tom.ReplicaContext; import bftsmart.tom.server.BatchExecutable; @@ -58,6 +58,9 @@ public abstract class DefaultRecoverable implements Recoverable, BatchExecutable private StateLog log; private StateManager stateManager; + /** + * Constructor + */ public DefaultRecoverable() { try { @@ -159,7 +162,7 @@ private byte[][] executeBatch(byte[][] commands, MessageContext[] msgCtxs, boole return replies; } - public final byte[] computeHash(byte[] data) { + private final byte[] computeHash(byte[] data) { byte[] ret = null; hashLock.lock(); ret = md.digest(data); @@ -410,9 +413,6 @@ public StateManager getStateManager() { return stateManager; } - - - @Override public byte[] executeUnordered(byte[] command, MessageContext msgCtx) { return appExecuteUnordered(command, msgCtx); @@ -430,12 +430,37 @@ public void noOp(int CID, byte[][] operations, MessageContext[] msgCtxs) { } + /** + * Given a snapshot received from the state transfer protocol, install it + * @param state The serialized snapshot + */ public abstract void installSnapshot(byte[] state); + /** + * Returns a serialized snapshot of the application state + * @return A serialized snapshot of the application state + */ public abstract byte[] getSnapshot(); + /** + * Execute a batch of ordered requests + * + * @param commands The batch of requests + * @param msgCtxs The context associated to each request + * @param fromConsensus true if the request arrived from a consensus execution, false if it arrives from the state transfer protocol + * + * @return the respective replies for each request + */ public abstract byte[][] appExecuteBatch(byte[][] commands, MessageContext[] msgCtxs, boolean fromConsensus); + /** + * Execute an unordered request + * + * @param command The unordered request + * @param msgCtx The context associated to the request + * + * @return the reply for the request issued by the client + */ public abstract byte[] appExecuteUnordered(byte[] command, MessageContext msgCtx); } diff --git a/src/bftsmart/tom/server/defaultservices/DefaultSingleRecoverable.java b/src/bftsmart/tom/server/defaultservices/DefaultSingleRecoverable.java index fc892fba8..416f725ea 100644 --- a/src/bftsmart/tom/server/defaultservices/DefaultSingleRecoverable.java +++ b/src/bftsmart/tom/server/defaultservices/DefaultSingleRecoverable.java @@ -26,7 +26,7 @@ import bftsmart.reconfiguration.util.TOMConfiguration; import bftsmart.statemanagement.ApplicationState; import bftsmart.statemanagement.StateManager; -import bftsmart.statemanagement.strategy.StandardStateManager; +import bftsmart.statemanagement.standard.StandardStateManager; import bftsmart.tom.MessageContext; import bftsmart.tom.ReplicaContext; import bftsmart.tom.server.Recoverable; @@ -109,7 +109,7 @@ private byte[] executeOrdered(byte[] command, MessageContext msgCtx, boolean noo return reply; } - public final byte[] computeHash(byte[] data) { + private final byte[] computeHash(byte[] data) { byte[] ret = null; hashLock.lock(); ret = md.digest(data); @@ -309,11 +309,35 @@ public void noOp(int CID, byte[][] operations, MessageContext[] msgCtx) { } } + /** + * Given a snapshot received from the state transfer protocol, install it + * @param state The serialized snapshot + */ public abstract void installSnapshot(byte[] state); + /** + * Returns a serialized snapshot of the application state + * @return A serialized snapshot of the application state + */ public abstract byte[] getSnapshot(); + /** + * Execute a batch of ordered requests + * + * @param command The ordered request + * @param msgCtx The context associated to each request + * + * @return the reply for the request issued by the client + */ public abstract byte[] appExecuteOrdered(byte[] command, MessageContext msgCtx); + /** + * Execute an unordered request + * + * @param command The unordered request + * @param msgCtx The context associated to the request + * + * @return the reply for the request issued by the client + */ public abstract byte[] appExecuteUnordered(byte[] command, MessageContext msgCtx); } diff --git a/src/bftsmart/tom/server/defaultservices/DiskStateLog.java b/src/bftsmart/tom/server/defaultservices/DiskStateLog.java index 618ec1e01..128ebc62c 100644 --- a/src/bftsmart/tom/server/defaultservices/DiskStateLog.java +++ b/src/bftsmart/tom/server/defaultservices/DiskStateLog.java @@ -55,6 +55,9 @@ public DiskStateLog(int id, byte[] initialState, byte[] initialHash, this.syncLog = syncLog; this.syncCkp = syncCkp; this.logPointers = new HashMap<>(); + + File directory = new File(DEFAULT_DIR); + if (!directory.exists()) directory.mkdir(); } private void createLogFile() { diff --git a/src/bftsmart/tom/server/defaultservices/FileRecoverer.java b/src/bftsmart/tom/server/defaultservices/FileRecoverer.java index f664b6b67..857dd0eb9 100644 --- a/src/bftsmart/tom/server/defaultservices/FileRecoverer.java +++ b/src/bftsmart/tom/server/defaultservices/FileRecoverer.java @@ -43,8 +43,8 @@ public class FileRecoverer { public FileRecoverer(int replicaId, String defaultDir) { this.replicaId = replicaId; this.defaultDir = defaultDir; - ckpLastConsensusId = 0; - logLastConsensusId = 0; + ckpLastConsensusId = -1; + logLastConsensusId = -1; } /** diff --git a/src/bftsmart/tom/server/defaultservices/durability/DurabilityCoordinator.java b/src/bftsmart/tom/server/durability/DurabilityCoordinator.java similarity index 92% rename from src/bftsmart/tom/server/defaultservices/durability/DurabilityCoordinator.java rename to src/bftsmart/tom/server/durability/DurabilityCoordinator.java index d668f8c76..644001dc3 100644 --- a/src/bftsmart/tom/server/defaultservices/durability/DurabilityCoordinator.java +++ b/src/bftsmart/tom/server/durability/DurabilityCoordinator.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.tom.server.defaultservices.durability; +package bftsmart.tom.server.durability; import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; @@ -23,9 +23,9 @@ import bftsmart.reconfiguration.util.TOMConfiguration; import bftsmart.statemanagement.ApplicationState; import bftsmart.statemanagement.StateManager; -import bftsmart.statemanagement.strategy.durability.CSTRequest; -import bftsmart.statemanagement.strategy.durability.CSTState; -import bftsmart.statemanagement.strategy.durability.DurableStateManager; +import bftsmart.statemanagement.durability.CSTRequest; +import bftsmart.statemanagement.durability.CSTState; +import bftsmart.statemanagement.durability.DurableStateManager; import bftsmart.tom.MessageContext; import bftsmart.tom.ReplicaContext; import bftsmart.tom.server.BatchExecutable; @@ -435,11 +435,35 @@ public void noOp(int CID, byte[][] operations, MessageContext[] msgCtxs) { } - public abstract void installSnapshot(byte[] state); + /** + * Given a snapshot received from the state transfer protocol, install it + * @param state The serialized snapshot + */ + public abstract void installSnapshot(byte[] state); - public abstract byte[] getSnapshot(); + /** + * Returns a serialized snapshot of the application state + * @return A serialized snapshot of the application state + */ + public abstract byte[] getSnapshot(); - public abstract byte[][] appExecuteBatch(byte[][] commands, MessageContext[] msgCtxs); + /** + * Execute a batch of ordered requests + * + * @param commands The batch of requests + * @param msgCtxs The context associated to each request + * + * @return the respective replies for each request + */ + public abstract byte[][] appExecuteBatch(byte[][] commands, MessageContext[] msgCtxs); - public abstract byte[] appExecuteUnordered(byte[] command, MessageContext msgCtx); + /** + * Execute an unordered request + * + * @param command The unordered request + * @param msgCtx The context associated to the request + * + * @return the reply for the request issued by the client + */ + public abstract byte[] appExecuteUnordered(byte[] command, MessageContext msgCtx); } diff --git a/src/bftsmart/tom/server/defaultservices/durability/DurableStateLog.java b/src/bftsmart/tom/server/durability/DurableStateLog.java similarity index 97% rename from src/bftsmart/tom/server/defaultservices/durability/DurableStateLog.java rename to src/bftsmart/tom/server/durability/DurableStateLog.java index 0c04a5d99..641c0b1a1 100644 --- a/src/bftsmart/tom/server/defaultservices/durability/DurableStateLog.java +++ b/src/bftsmart/tom/server/durability/DurableStateLog.java @@ -13,7 +13,7 @@ See the License for the specific language governing permissions and limitations under the License. */ -package bftsmart.tom.server.defaultservices.durability; +package bftsmart.tom.server.durability; import java.io.ByteArrayOutputStream; import java.io.File; @@ -27,9 +27,9 @@ import java.util.Map; import java.util.concurrent.locks.ReentrantLock; -import bftsmart.statemanagement.strategy.durability.CSTRequest; -import bftsmart.statemanagement.strategy.durability.CSTRequestF1; -import bftsmart.statemanagement.strategy.durability.CSTState; +import bftsmart.statemanagement.durability.CSTRequest; +import bftsmart.statemanagement.durability.CSTRequestF1; +import bftsmart.statemanagement.durability.CSTState; import bftsmart.tom.MessageContext; import bftsmart.tom.server.defaultservices.CommandsInfo; import bftsmart.tom.server.defaultservices.FileRecoverer; @@ -62,6 +62,10 @@ public DurableStateLog(int id, byte[] initialState, byte[] initialHash, this.syncLog = syncLog; this.syncCkp = syncCkp; this.logPointers = new HashMap(); + + File directory = new File(DEFAULT_DIR); + if (!directory.exists()) directory.mkdir(); + this.fr = new FileRecoverer(id, DEFAULT_DIR); } diff --git a/src/bftsmart/tom/util/BatchBuilder.java b/src/bftsmart/tom/util/BatchBuilder.java index 033e0028d..677eaf1d5 100644 --- a/src/bftsmart/tom/util/BatchBuilder.java +++ b/src/bftsmart/tom/util/BatchBuilder.java @@ -46,10 +46,24 @@ public BatchBuilder(long seed){ /** build buffer */ private byte[] createBatch(long timestamp, int numberOfNonces, long seed, int numberOfMessages, int totalMessagesSize, - boolean useSignatures, byte[][] messages, byte[][] signatures, ServerViewController controller) { + boolean useSignatures, byte[][] messages, byte[][] signatures) { + + int sigsSize = 0; + + if (useSignatures) { + + sigsSize = Integer.BYTES * numberOfMessages; + + for (byte[] sig : signatures) { + + sigsSize += sig.length; + } + } + int size = 20 + //timestamp 8, nonces 4, nummessages 4 (numberOfNonces > 0 ? 8 : 0) + //seed if needed - (numberOfMessages*(4+(useSignatures?TOMUtil.getSignatureSize(controller):0)))+ // msglength + signature for each msg + (Integer.BYTES * numberOfMessages) + // messages length + sigsSize + // signatures size totalMessagesSize; //size of all msges ByteBuffer proposalBuffer = ByteBuffer.allocate(size); @@ -65,22 +79,27 @@ private byte[] createBatch(long timestamp, int numberOfNonces, long seed, int nu proposalBuffer.putInt(numberOfMessages); for (int i = 0; i < numberOfMessages; i++) { - putMessage(proposalBuffer,messages[i], false, signatures[i]); + putMessage(proposalBuffer,messages[i], useSignatures, signatures[i]); } return proposalBuffer.array(); } - private void putMessage(ByteBuffer proposalBuffer, byte[] message, boolean isHash, byte[] signature) { - proposalBuffer.putInt(isHash?0:message.length); + private void putMessage(ByteBuffer proposalBuffer, byte[] message, boolean addSig, byte[] signature) { + proposalBuffer.putInt(message.length); proposalBuffer.put(message); - if(signature != null) { - proposalBuffer.put(signature); - } + if (addSig) { + if(signature != null) { + proposalBuffer.putInt(signature.length); + proposalBuffer.put(signature); + } else { + proposalBuffer.putInt(0); + } + } } - public byte[] makeBatch(List msgs, int numNounces, long timestamp, ServerViewController controller) { + public byte[] makeBatch(List msgs, int numNounces, long timestamp, boolean useSignatures) { int numMsgs = msgs.size(); int totalMessageSize = 0; //total size of the messages being batched @@ -103,10 +122,10 @@ public byte[] makeBatch(List msgs, int numNounces, long timestamp, S // return the batch return createBatch(timestamp, numNounces,rnd.nextLong(), numMsgs, totalMessageSize, - controller.getStaticConf().getUseSignatures() == 1, messages, signatures, controller); + useSignatures, messages, signatures); } - public byte[] makeBatch(List msgs, int numNounces, long seed, long timestamp, ServerViewController controller) { + public byte[] makeBatch(List msgs, int numNounces, long seed, long timestamp, boolean useSignatures) { int numMsgs = msgs.size(); int totalMessageSize = 0; //total size of the messages being batched @@ -129,7 +148,7 @@ public byte[] makeBatch(List msgs, int numNounces, long seed, long t // return the batch return createBatch(timestamp, numNounces,seed, numMsgs, totalMessageSize, - controller.getStaticConf().getUseSignatures() == 1, messages, signatures, controller); + useSignatures, messages, signatures); } } diff --git a/src/bftsmart/tom/util/BatchReader.java b/src/bftsmart/tom/util/BatchReader.java index 8562090e4..753b25443 100644 --- a/src/bftsmart/tom/util/BatchReader.java +++ b/src/bftsmart/tom/util/BatchReader.java @@ -68,10 +68,17 @@ public TOMMessage[] deserialiseRequests(ServerViewController controller) { proposalBuffer.get(message); byte[] signature = null; - if(useSignatures){ - signature = new byte[TOMUtil.getSignatureSize(controller)]; - proposalBuffer.get(signature); + + if (useSignatures) { + + int sigSize = proposalBuffer.getInt(); + + if (sigSize > 0) { + signature = new byte[sigSize]; + proposalBuffer.get(signature); + } } + //obtain the nonces to be delivered to the application byte[] nonces = new byte[numberOfNonces]; if (nonces.length > 0) { diff --git a/src/bftsmart/tom/util/ECDSAKeyPairGenerator.java b/src/bftsmart/tom/util/ECDSAKeyPairGenerator.java new file mode 100644 index 000000000..e344c7430 --- /dev/null +++ b/src/bftsmart/tom/util/ECDSAKeyPairGenerator.java @@ -0,0 +1,97 @@ +/** +Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ +package bftsmart.tom.util; + +import bftsmart.reconfiguration.util.TOMConfiguration; +import java.io.BufferedWriter; +import java.io.FileWriter; +import java.security.Key; +import java.security.KeyPair; +import java.security.KeyPairGenerator; +import java.security.PrivateKey; +import java.security.PublicKey; +import java.security.Security; +import java.security.spec.ECGenParameterSpec; +import org.apache.commons.codec.binary.Base64; + +/** + * Utility class used to generate a key pair for some process id on + * config/keys/publickey and config/keys/privatekey + * + */ +public class ECDSAKeyPairGenerator { + + /** Creates a new instance of KeyPairGenerator */ + public ECDSAKeyPairGenerator() { + } + + /** + * Generate the key pair for the process with id = and put it on the + * files config/keys/publickey and config/keys/privatekey + * + * @param id the id of the process to generate key + * @throws Exception something goes wrong when writing the files + */ + public void run(int id, String domainParam, String provider) throws Exception { + + + ECGenParameterSpec specs = new ECGenParameterSpec(domainParam); + KeyPairGenerator keyGen = KeyPairGenerator.getInstance("EC",provider); + keyGen.initialize(specs); + + KeyPair kp = keyGen.generateKeyPair(); + PublicKey puk = kp.getPublic(); + PrivateKey prk = kp.getPrivate(); + saveToFile(id,puk,prk); + } + + private void saveToFile(int id, PublicKey puk, PrivateKey prk) throws Exception { + String path = "config"+System.getProperty("file.separator")+"ecdsakeys"+ + System.getProperty("file.separator"); + + BufferedWriter w = new BufferedWriter(new FileWriter(path+"publickey"+id,false)); + w.write(getKeyAsString(puk)); + w.flush(); + w.close(); + + w = new BufferedWriter(new FileWriter(path+"privatekey"+id,false)); + w.write(getKeyAsString(prk)); + w.flush(); + w.close(); + } + + + private String getKeyAsString(Key key) { + byte[] keyBytes = key.getEncoded(); + + return Base64.encodeBase64String(keyBytes); + } + + public static void main(String[] args) throws Exception{ + + if (args.length < 2) System.err.println("Use: ECDSAKeyPairGenerator [config dir]"); + String confHome = ""; + if (args.length > 2) confHome = args[2]; + + TOMConfiguration conf = new TOMConfiguration(Integer.parseInt(args[0]), confHome, null); + String provider = conf.getSignatureAlgorithmProvider(); + + + new ECDSAKeyPairGenerator().run(Integer.parseInt(args[0]), args[1], provider); + + } + +} diff --git a/src/bftsmart/tom/util/Extractor.java b/src/bftsmart/tom/util/Extractor.java index c84c93bfa..b32b69891 100644 --- a/src/bftsmart/tom/util/Extractor.java +++ b/src/bftsmart/tom/util/Extractor.java @@ -21,8 +21,16 @@ * Provides support for building custom response extractors to be used in the * ServiceProxy. * - * @author alysson */ public interface Extractor { + + /** + * Extracts a reply given a set of replies from a set of replicas. + * + * @param replies Set of replies from a set of replicas. + * @param sameContent Whether or not the replies are supposed to have the same content + * @param lastReceived Last reply received from the replicas. This is an index in relation to the `replies` parameter. + * @return + */ TOMMessage extractResponse(TOMMessage[] replies, int sameContent, int lastReceived); } diff --git a/src/bftsmart/tom/util/KeyLoader.java b/src/bftsmart/tom/util/KeyLoader.java index e41feef1b..eb231c425 100644 --- a/src/bftsmart/tom/util/KeyLoader.java +++ b/src/bftsmart/tom/util/KeyLoader.java @@ -14,14 +14,52 @@ import java.security.spec.InvalidKeySpecException; /** - * - * @author joao + * The KeyLoader interface is used internally by BFT-SMaRt to load signature keys from disk. Developers can use implementations of this + * interface so that if their application uses key structures similar to the library, these keys can be shared by both the library and + * the application to simplify key management. */ public interface KeyLoader { + /** + * Fetches the public key for the specified replica/client. + * + * @param id ID of the respective replica/client. + * @return The replica/client's public key. + * + * @throws IOException If there was an error reading the key file. + * @throws NoSuchAlgorithmException If the algorithm specified in the configuration does not exist. + * @throws InvalidKeySpecException If the key specs are not properly set. + * @throws CertificateException If there was an error loading the public key certificate. + */ public PublicKey loadPublicKey(int id) throws IOException, NoSuchAlgorithmException, InvalidKeySpecException, CertificateException; + + /** + * Fetches the public key for this replica/client. + * + * @return The replica/client's public key. + * + * @throws IOException If there was an error reading the key file. + * @throws NoSuchAlgorithmException If the algorithm specified by the key does not exist. + * @throws InvalidKeySpecException If the key specs are not properly set. + * @throws CertificateException If there was an error loading the public key certificate. + */ public PublicKey loadPublicKey() throws IOException, NoSuchAlgorithmException, InvalidKeySpecException, CertificateException; + + /** + * Fetches the private key for this replica/client. + * + * @return The replica/client's public key. + * + * @throws IOException If there was an error reading the key file. + * @throws NoSuchAlgorithmException If the algorithm specified in the configuration does not exist. + * @throws InvalidKeySpecException If the key specs are not properly set . + */ public PrivateKey loadPrivateKey() throws IOException, NoSuchAlgorithmException, InvalidKeySpecException; + + /** + * Get the signature algorithm specified by the key. + * @return The signature algorithm specified by the key. + */ public String getSignatureAlgorithm(); } diff --git a/src/bftsmart/tom/util/RSAKeyPairGenerator.java b/src/bftsmart/tom/util/RSAKeyPairGenerator.java index 7ff398478..9ea4b84fb 100644 --- a/src/bftsmart/tom/util/RSAKeyPairGenerator.java +++ b/src/bftsmart/tom/util/RSAKeyPairGenerator.java @@ -15,6 +15,7 @@ */ package bftsmart.tom.util; +import bftsmart.reconfiguration.util.TOMConfiguration; import java.io.BufferedWriter; import java.io.FileWriter; import java.security.Key; @@ -42,8 +43,8 @@ public RSAKeyPairGenerator() { * @param id the id of the process to generate key * @throws Exception something goes wrong when writing the files */ - public void run(int id, int size) throws Exception { - KeyPairGenerator keyGen = KeyPairGenerator.getInstance("RSA"); + public void run(int id, int size, String provider) throws Exception { + KeyPairGenerator keyGen = KeyPairGenerator.getInstance("RSA", provider); keyGen.initialize(size); KeyPair kp = keyGen.generateKeyPair(); PublicKey puk = kp.getPublic(); @@ -73,12 +74,18 @@ private String getKeyAsString(Key key) { return Base64.encodeBase64String(keyBytes); } - public static void main(String[] args){ - try{ - new RSAKeyPairGenerator().run(Integer.parseInt(args[0]), Integer.parseInt(args[1])); - }catch(Exception e){ - System.err.println("Use: RSAKeyPairGenerator "); - } + public static void main(String[] args) throws Exception{ + + if (args.length < 2) System.err.println("Use: RSAKeyPairGenerator [config dir]"); + String confHome = ""; + if (args.length > 2) confHome = args[2]; + + TOMConfiguration conf = new TOMConfiguration(Integer.parseInt(args[0]), confHome, null); + String provider = conf.getSignatureAlgorithmProvider(); + + + new RSAKeyPairGenerator().run(Integer.parseInt(args[0]), Integer.parseInt(args[1]), provider); + } } diff --git a/src/bftsmart/tom/util/TOMUtil.java b/src/bftsmart/tom/util/TOMUtil.java index a89261700..4c77e7207 100644 --- a/src/bftsmart/tom/util/TOMUtil.java +++ b/src/bftsmart/tom/util/TOMUtil.java @@ -28,19 +28,18 @@ import java.security.SignatureException; import java.util.Arrays; -import bftsmart.reconfiguration.ViewController; import bftsmart.reconfiguration.util.Configuration; -import java.security.Provider; -import javax.crypto.Mac; +import java.security.Security; +import java.util.Random; import javax.crypto.SecretKeyFactory; -import org.bouncycastle.jce.provider.BouncyCastleProvider; +import javax.crypto.spec.PBEKeySpec; +import org.bouncycastle.jce.provider.BouncyCastleProvider; import org.slf4j.Logger; import org.slf4j.LoggerFactory; public class TOMUtil { - //private static final int BENCHMARK_PERIOD = 10000; private static Logger logger = LoggerFactory.getLogger(TOMUtil.class); //some message types @@ -58,44 +57,50 @@ public class TOMUtil { public static final int TRIGGER_LC_LOCALLY = 8; public static final int TRIGGER_SM_LOCALLY = 9; - private static int signatureSize = -1; private static boolean init = false; + + private static String secretAlgorithm = Configuration.DEFAULT_SECRETKEY; + private static String secretAlgorithmProvider = Configuration.DEFAULT_SECRETKEY_PROVIDER; - private static Provider provider = new BouncyCastleProvider(); + private static String hashAlgorithm = Configuration.DEFAULT_HASH; + private static String hashAlgorithmProvider = Configuration.DEFAULT_HASH_PROVIDER; - private static String hmacAlgorithm = Configuration.DEFAULT_HMAC; - private static String secretAlgorithm = Configuration.DEFAULT_SECRETKEY; private static String sigAlgorithm = Configuration.DEFAULT_SIGNATURE; - private static String hashAlgorithm = Configuration.DEFAULT_HASH; - - public static void init(Provider provider, String hmacAlgorithm, String secretAlgorithm, String sigAlgorithm, String hashAlgorithm) { + private static String sigAlgorithmProvider = Configuration.DEFAULT_SIGNATURE_PROVIDER; + + private static final int SALT_SEED = 509; + private static final int SALT_BYTE_SIZE = 64; // 512 bits + private static final int HASH_BYTE_SIZE = 64; // 512 bits + private static final int PBE_ITERATIONS = 1000; + + public static void init( + String secretAlgorithm, + String sigAlgorithm, + String hashAlgorithm, + String secretAlgorithmProvider, + String sigAlgorithmProvider, + String hashAlgorithmProvider) { if (!TOMUtil.init) { - - TOMUtil.hmacAlgorithm = hmacAlgorithm; - TOMUtil.sigAlgorithm = sigAlgorithm; + TOMUtil.secretAlgorithm = secretAlgorithm; + TOMUtil.secretAlgorithmProvider = secretAlgorithmProvider; + TOMUtil.hashAlgorithm = hashAlgorithm; - + TOMUtil.hashAlgorithmProvider = hashAlgorithmProvider; + + TOMUtil.sigAlgorithm = sigAlgorithm; + TOMUtil.sigAlgorithmProvider = sigAlgorithmProvider; + TOMUtil.init = true; + + if(TOMUtil.sigAlgorithmProvider.equals("BC")) { + logger.info("Including BouncyCastle as Signature Provider."); + Security.addProvider(new BouncyCastleProvider()); + } } } - public static int getSignatureSize(ViewController controller) { - if (signatureSize > 0) { - return signatureSize; - } - - byte[] signature = signMessage(controller.getStaticConf().getPrivateKey(), - "a".getBytes()); - - if (signature != null) { - signatureSize = signature.length; - } - - return signatureSize; - } - //******* EDUARDO BEGIN **************// public static byte[] getBytes(Object o) { ByteArrayOutputStream bOut = new ByteArrayOutputStream(); @@ -228,21 +233,27 @@ public static final byte[] computeHash(byte[] data) { public static Signature getSigEngine() throws NoSuchAlgorithmException { - return Signature.getInstance(TOMUtil.sigAlgorithm, TOMUtil.provider); + return Signature.getInstance(TOMUtil.sigAlgorithm, Security.getProvider(TOMUtil.sigAlgorithmProvider)); } public static MessageDigest getHashEngine() throws NoSuchAlgorithmException { - return MessageDigest.getInstance(TOMUtil.hashAlgorithm, TOMUtil.provider); + return MessageDigest.getInstance(TOMUtil.hashAlgorithm, Security.getProvider(TOMUtil.hashAlgorithmProvider)); } public static SecretKeyFactory getSecretFactory() throws NoSuchAlgorithmException { - return SecretKeyFactory.getInstance(TOMUtil.secretAlgorithm, TOMUtil.provider); + return SecretKeyFactory.getInstance(TOMUtil.secretAlgorithm, Security.getProvider(TOMUtil.secretAlgorithmProvider)); } - public static Mac getMacFactory() throws NoSuchAlgorithmException { + public static PBEKeySpec generateKeySpec(char[] password) throws NoSuchAlgorithmException { + + // generate salt + Random random = new Random(SALT_SEED); + byte salt[] = new byte[SALT_BYTE_SIZE]; + random.nextBytes(salt); + + return new PBEKeySpec(password, salt, PBE_ITERATIONS, HASH_BYTE_SIZE); - return Mac.getInstance(TOMUtil.hmacAlgorithm, TOMUtil.provider); } } diff --git a/test/bftsmart/ConsoleLogger.java b/test/bftsmart/ConsoleLogger.java deleted file mode 100644 index 0b072a169..000000000 --- a/test/bftsmart/ConsoleLogger.java +++ /dev/null @@ -1,65 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart; - -import java.io.BufferedReader; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.io.PrintStream; - -/** - * - * @author Marcel Santos - * - */ -public class ConsoleLogger extends Thread { - - private InputStream in = null; - private PrintStream out = null; - private String index; - - public InputStream getIn() { - return in; - } - public void setIn(InputStream in) { - this.in = in; - } - public PrintStream getOut() { - return out; - } - public void setOut(PrintStream out) { - this.out = out; - } - public String getIndex() { - return index; - } - public void setIndex(String index) { - this.index = index; - } - - public void run() { - BufferedReader stdInput = new BufferedReader(new InputStreamReader(in)); - String s; - try { - while ((s = stdInput.readLine()) != null) { - out.println("Replica " + index + ")" +s); - } - } catch(IOException ioe) { - System.out.println("----------- Exception writing replica log: " + ioe.getMessage()); - } - } -} diff --git a/test/bftsmart/TestFixture.java b/test/bftsmart/TestFixture.java deleted file mode 100644 index 6dc6f29cd..000000000 --- a/test/bftsmart/TestFixture.java +++ /dev/null @@ -1,153 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart; - -import java.io.IOException; - -import org.junit.AfterClass; -import org.junit.BeforeClass; - -/** - * - * @author Marcel Santos - * - */ -public class TestFixture { - - private static Process replica0; - private static Process replica1; - private static Process replica2; - private static Process replica3; - - private static ConsoleLogger log0; - private static ConsoleLogger log1; - private static ConsoleLogger log2; - private static ConsoleLogger log3; - - private static String[] command = new String[5]; - - @BeforeClass - public static void startServers() { - try { - System.out.println("Stoping running servers, if any"); - String stopCmd = "kill -9 $(ps aux | grep '[b]ftsmart.demo.bftmap.BFTMapServer' | awk '{print $ 2}')"; - try { - new ProcessBuilder(stopCmd).redirectErrorStream(true).start(); - } catch (IOException e) { - System.out.println("Exception stoping remaining replicas"); - e.printStackTrace(); - } - - System.out.println("Starting the servers"); - command[0] = "java"; - command[1] = "-cp"; - command[2] = "bin/:lib/*"; - command[3] = "bftsmart.demo.bftmap.BFTMapServer"; - command[4] = "0"; - - startServer(0); - - Thread.sleep(2000); - startServer(1); - - Thread.sleep(2000); - startServer(2); - - Thread.sleep(2000); - startServer(3); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - @AfterClass - public static void stopServers() throws IllegalArgumentException, IllegalAccessException, NoSuchFieldException, IOException { - System.out.println("Stopping servers"); - stopServer(0); - stopServer(1); - stopServer(2); - stopServer(3); - System.out.println("Servers stopped"); - } - - public static void stopServer(int id) { - switch(id) { - case 0: - replica0.destroy(); - break; - case 1: - replica1.destroy(); - break; - case 2: - replica2.destroy(); - break; - case 3: - replica3.destroy(); - break; - default: - System.out.println("### Couldn't stop server. Server not found ###"); - break; - } - } - - public static void startServer(int id) { - command[4] = String.valueOf(id); - try { - switch(id) { - case 0: - replica0 = new ProcessBuilder(command).redirectErrorStream(true).start(); - log0 = new ConsoleLogger(); - log0.setIn(replica0.getInputStream()); - log0.setOut(System.out); - log0.setIndex(String.valueOf(id)); - log0.start(); - break; - case 1: - replica1 = new ProcessBuilder(command).redirectErrorStream(true).start(); - log1 = new ConsoleLogger(); - log1.setIn(replica1.getInputStream()); - log1.setOut(System.out); - log1.setIndex(String.valueOf(id)); - log1.start(); - break; - case 2: - replica2 = new ProcessBuilder(command).redirectErrorStream(true).start(); - log2 = new ConsoleLogger(); - log2.setIn(replica2.getInputStream()); - log2.setOut(System.out); - log2.setIndex(String.valueOf(id)); - log2.start(); - break; - case 3: - replica3 = new ProcessBuilder(command).redirectErrorStream(true).start(); - log3 = new ConsoleLogger(); - log3.setIn(replica3.getInputStream()); - log3.setOut(System.out); - log3.setIndex(String.valueOf(id)); - log3.start(); - break; - default: - System.out.println("Id not supported"); - break; - } - } catch (IOException e) { - // TODO Auto-generated catch block - e.printStackTrace(); - } - } - -} diff --git a/test/bftsmart/demo/bftmap/BFTMapClientTest.java b/test/bftsmart/demo/bftmap/BFTMapClientTest.java deleted file mode 100644 index 3e4e39cc3..000000000 --- a/test/bftsmart/demo/bftmap/BFTMapClientTest.java +++ /dev/null @@ -1,261 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.demo.bftmap; - -import static org.junit.Assert.*; - -import java.util.HashMap; - -import org.junit.Test; - -import bftsmart.demo.bftmap.BFTMap; -import bftsmart.TestFixture; - -/** - * - * @author Marcel Santos - * - */ -public class BFTMapClientTest extends TestFixture { - - private BFTMap bftMap; - - private void insert(String tableName, int entries) { - int index = bftMap.size1(tableName); - for(int i = 0; i < entries; i++) { - String key = "key" + (i + index); - String value = "value" + (i + index); - bftMap.putEntry(tableName, key, value.getBytes()); - } - } - - /** - * Test regular case where there is the creation of a table, insert of - * data and search for size of the table. - * No servers are killed nor behaves different than expected. - */ - @Test - public void testRegularCase() { - try{ - Thread.sleep(1000); - bftMap = new BFTMap(1001); - bftMap.put("TestTable1", new HashMap()); - bftMap.putEntry("TestTable1", "key0", "value0".getBytes()); - assertEquals("Main table size should be 1", 1, bftMap.size1("TestTable1")); - - insert("TestTable1", 100); - - assertEquals("Main table size should be 101", 101, bftMap.size1("TestTable1")); - - bftMap.putEntry("TestTable1", "key102", "value102".getBytes()); - assertEquals("Main table size should be 102", 102, bftMap.size1("TestTable1")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - /** - * - Crate table; - * - Insert data; - * - Kills a replica that is not a leader; - * - Insert data; - * - Verify if the size of the table is correct. - */ - @Test - public void testStopNonLeader() { - try{ - Thread.sleep(1000); - bftMap = new BFTMap(1001); - bftMap.put("TestTable2", new HashMap()); - insert("TestTable2", 1); - assertEquals("Main table size should be 1", 1, bftMap.size1("TestTable2")); - - insert("TestTable2", 200); - assertEquals("Main table size should be 201", 201, bftMap.size1("TestTable2")); - - stopServer(1); - insert("TestTable2", 200); - assertEquals("Main table size should be 401", 401, bftMap.size1("TestTable2")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - /** - * This test insert and retrieve data. - * During this process a replica that is not the leader is killed. - * After that the replica is started back. - * During the whole process messages keep being sent, to test if - * the application works as expected. - */ - @Test - public void testStopAndStartNonLeader() { - try{ - Thread.sleep(5000); - bftMap = new BFTMap(1001); - Thread.sleep(1000); - bftMap.put("TestTable3", new HashMap()); - - insert("TestTable3", 65); - assertEquals("Main table size should be 65", 65, bftMap.size1("TestTable3")); - - stopServer(1); - insert("TestTable3", 35); - assertEquals("Main table size should be 100", 100, bftMap.size1("TestTable3")); - - startServer(1); - - Thread.sleep(10000); - - insert("TestTable3", 35); - assertEquals("Main table size should be 135", 135, bftMap.size1("TestTable3")); - - Thread.sleep(10000); - - stopServer(2); - - insert("TestTable3", 35); - assertEquals("Main table size should be 170", 170, bftMap.size1("TestTable3")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - /** - * - Crate table; - * - Insert data; - * - Kills the leader replica; - * - Insert data; - * - Verify if the size of the table is correct. - */ - @Test - public void testStopLeader() { - try{ - Thread.sleep(1000); - bftMap = new BFTMap(1001); - Thread.sleep(1000); - bftMap.put("TestTable4", new HashMap()); - insert("TestTable4", 1); - assertEquals("Main table size should be 1", 1, bftMap.size1("TestTable4")); - - insert("TestTable4", 200); - assertEquals("Main table size should be 201", 201, bftMap.size1("TestTable4")); - - stopServer(0); - - Thread.sleep(1000); - - insert("TestTable4", 200); - assertEquals("Main table size should be 401", 401, bftMap.size1("TestTable4")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - - /** - * - Crate table; - * - Insert data; - * - Kills the leader replica; - * - Insert data; - * - Verify if the size of the table is correct; - * - Start the replica again; - * - Insert data; - * - Verify if the size of the table is correct; - * - Kills a second leader; - * - Insert data; - * - Verify if the size of the table is correct. - */ - @Test - public void testStopLeaders() { - try{ - Thread.sleep(1000); - bftMap = new BFTMap(1001); - bftMap.put("TestTable5", new HashMap()); - - insert("TestTable5", 130); - assertEquals("Main table size should be 130", 130, bftMap.size1("TestTable5")); - - System.out.println("stopping server 0"); - stopServer(0); - System.out.println("stoped"); - - insert("TestTable5", 60); - assertEquals("Main table size should be 190", 190, bftMap.size1("TestTable5")); - - System.out.println("starting server 0"); - startServer(0); - System.out.println("started"); - - Thread.sleep(10000); - - insert("TestTable5", 20); - assertEquals("Main table size should be 210", 210, bftMap.size1("TestTable5")); - - Thread.sleep(10000); - - insert("TestTable5", 20); - assertEquals("Main table size should be 230", 230, bftMap.size1("TestTable5")); - - System.out.println("stopping server 2"); - stopServer(2); - System.out.println("stopped"); - - insert("TestTable5", 40); - assertEquals("Main table size should be 270", 270, bftMap.size1("TestTable5")); - - startServer(2); - insert("TestTable5", 20); - assertEquals("Main table size should be 290", 290, bftMap.size1("TestTable5")); - - System.out.println("stopping server 3"); - stopServer(3); - System.out.println("stopped"); - - insert("TestTable5", 40); - assertEquals("Main table size should be 330", 330, bftMap.size1("TestTable5")); - - System.out.println("starting server 0"); - startServer(3); - System.out.println("started"); - - insert("TestTable5", 40); - assertEquals("Main table size should be 370", 370, bftMap.size1("TestTable5")); - - System.out.println("stopping server 1"); - stopServer(1); - System.out.println("stopped"); - - insert("TestTable5", 40); - assertEquals("Main table size should be 410", 410, bftMap.size1("TestTable5")); - - System.out.println("starting server 1"); - startServer(1); - System.out.println("started"); - - insert("TestTable5", 20); - assertEquals("Main table size should be 430", 430, bftMap.size1("TestTable5")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - -} diff --git a/test/bftsmart/demo/bftmap/ConsoleTest.java b/test/bftsmart/demo/bftmap/ConsoleTest.java deleted file mode 100644 index 838e38f47..000000000 --- a/test/bftsmart/demo/bftmap/ConsoleTest.java +++ /dev/null @@ -1,221 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.demo.bftmap; - -import static org.junit.Assert.*; - -import java.io.IOException; -import java.util.HashMap; - - -import org.junit.AfterClass; -import org.junit.BeforeClass; -import org.junit.Test; - -import bftsmart.ConsoleLogger; -import bftsmart.demo.bftmap.BFTMap; - -/** - * - * @author Marcel Santos - * - */ -public class ConsoleTest { - - // Comment to test SVN branch creation left by Marcel - private static Process replica0; - private static Process replica1; - private static Process replica2; - private static Process replica3; - private static String[] command = new String[5]; - - @BeforeClass - public static void startServers() { - try { - System.out.println("Starting the servers"); - command[0] = "java"; - command[1] = "-cp"; - command[2] = "bin/SMaRt.jar:lib/slf4j-api-1.5.8.jar:lib/slf4j-jdk14-1.5.8.jar:lib/netty-3.1.1.GA.jar:lib/commons-codec-1.5.jar"; - command[3] = "bftsmart.demo.keyvalue.BFTMapImpl"; - command[4] = "0"; - replica0 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log0 = new ConsoleLogger(); - log0.setIndex("0"); - log0.setIn(replica0.getInputStream()); - log0.setOut(System.out); - log0.start(); - - command[4] = "1"; - replica1 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log1 = new ConsoleLogger(); - log1.setIndex("1"); - log1.setIn(replica1.getInputStream()); - log1.setOut(System.out); - log1.start(); - - command[4] = "2"; - replica2 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log2 = new ConsoleLogger(); - log2.setIndex("2"); - log2.setIn(replica2.getInputStream()); - log2.setOut(System.out); - log2.start(); - - command[4] = "3"; - replica3 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log3 = new ConsoleLogger(); - log3.setIndex("3"); - log3.setIn(replica3.getInputStream()); - log3.setOut(System.out); - log3.start(); - - System.out.println("Servers started"); - - } catch(IOException ioe) { - System.out.println("Exception during BFTMapInteractiveClient test: "); - System.out.println(ioe.getMessage()); - } - } - - @AfterClass - public static void stopServers() throws IllegalArgumentException, IllegalAccessException, NoSuchFieldException, IOException { - System.out.println("Stopping servers"); - replica0.destroy(); - replica1.destroy(); - replica2.destroy(); - replica3.destroy(); - System.out.println("Servers stopped"); - } - - /** - * Test regular case where there is the creation of a table, insert of - * data and search for size of the table. - * No servers are killed nor behaves different than expected. - */ - @Test - public void testRegularCase() { - try{ - Thread.sleep(1000); - BFTMap bftMap = new BFTMap(1001); - bftMap.put("TestTable1", new HashMap()); - bftMap.putEntry("TestTable1", "key1", "value1".getBytes()); - assertEquals("Main table size should be 1", 1, bftMap.size1("TestTable1")); - - for(int i = 0; i < 100; i++) { - String key = "key" + (2+i); - String value = "value" + (2+i); - bftMap.putEntry("TestTable1", key, value.getBytes()); - } - assertEquals("Main table size should be 101", 101, bftMap.size1("TestTable1")); - - bftMap.putEntry("TestTable1", "key102", "value102".getBytes()); - assertEquals("Main table size should be 102", 102, bftMap.size1("TestTable1")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - public static void main(String[] args) { - ConsoleTest test = new ConsoleTest(); - startServers(); - test.testStopAndStartNonLeader(); - try { - stopServers(); - } catch(Exception e) { - e.printStackTrace(); - } - } - - /** - * This test insert and retrieve data. - * During this process a replica that is not the leader is killed. - * After that the replica is started back. - * During the whole process messages keep being sent, to test if - * the application works as expected. - */ - @Test - public void testStopAndStartNonLeader() { - try{ - Thread.sleep(1000); - BFTMap bftMap = new BFTMap(1001); - bftMap.put("TestTable2", new HashMap()); - - for(int i = 0; i < 65; i++) { - String key = "key" + (1+i); - String value = "value" + (2+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - - System.out.println("---------Sleep1: " + new java.util.Date()); - Thread.sleep(5000); - System.out.println("---------Wakeup1: " + new java.util.Date()); - - System.out.println("---------Killing replica 1"); - replica1.destroy(); // Killing a non-leader replica, replica1 - - System.out.println("---------Sleep2: " + new java.util.Date()); - Thread.sleep(5000); - System.out.println("---------Wakeup2: " + new java.util.Date()); - - for(int i = 0; i < 35; i++) { - String key = "key" + (66+i); - String value = "value" + (66+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - - System.out.println("---------Sleep3: " + new java.util.Date()); - Thread.sleep(5000); - System.out.println("---------Wakeup3: " + new java.util.Date()); - - command[4] = "1"; - System.out.println("---------Starting replica 1"); - replica1 = new ProcessBuilder(command).start(); // Starting replica1 back - ConsoleLogger log1 = new ConsoleLogger(); - log1.setIndex("11"); - log1.setIn(replica1.getInputStream()); - log1.setOut(System.out); - log1.start(); - - System.out.println("---------Sleep4: " + new java.util.Date()); - Thread.sleep(20000); - System.out.println("---------Wakeup4: " + new java.util.Date()); - - for(int i = 0; i < 35; i++) { - String key = "key" + (101+i); - String value = "value" + (101+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - - System.out.println("---------Sleep2: " + new java.util.Date()); - Thread.sleep(20000); - System.out.println("---------Wakeup2: " + new java.util.Date()); - - replica2.destroy(); // Killing another non-leader replica, replica3 - for(int i = 0; i < 35; i++) { - String key = "key" + (136+i); - String value = "value" + (136+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - System.out.println("----------Table size:" + bftMap.size1("TestTable2")); - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } catch(IOException ioe) { - System.out.println("Exception when starting replica 2: " + ioe.getMessage()); - } - } - -} diff --git a/test/bftsmart/demo/bftmap/LogWriter.java b/test/bftsmart/demo/bftmap/LogWriter.java deleted file mode 100644 index d3212783c..000000000 --- a/test/bftsmart/demo/bftmap/LogWriter.java +++ /dev/null @@ -1,70 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.demo.bftmap; - -import java.io.BufferedReader; -import java.io.FileWriter; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.io.PrintStream; -import java.io.PrintWriter; - -/** - * - * @author Marcel Santos - * - */ -public class LogWriter extends Thread { - - private InputStream in = null; - private PrintStream out = null; - private int index; - - public InputStream getIn() { - return in; - } - public void setIn(InputStream in) { - this.in = in; - } - public PrintStream getOut() { - return out; - } - public void setOut(PrintStream out) { - this.out = out; - } - public int getIndex() { - return index; - } - public void setIndex(int index) { - this.index = index; - } - - public void run() { - BufferedReader stdInput = new BufferedReader(new InputStreamReader(in)); - String s; - try { - PrintWriter pw = new PrintWriter(new FileWriter("ServerLog-" + index + ".debug")); - while ((s = stdInput.readLine()) != null) { -// out.println(s); - pw.println(s); - } - pw.close(); - } catch(IOException ioe) { - System.out.println("----------- Exception writing replica log: " + ioe.getMessage()); - } - } -} diff --git a/test/bftsmart/demo/bftmapjunit/ConsoleLogger.java b/test/bftsmart/demo/bftmapjunit/ConsoleLogger.java deleted file mode 100644 index 6e14419c7..000000000 --- a/test/bftsmart/demo/bftmapjunit/ConsoleLogger.java +++ /dev/null @@ -1,66 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.demo.bftmapjunit; - -import java.io.BufferedReader; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.io.PrintStream; - -/** - * - * @author Marcel Santos - * - */ -public class ConsoleLogger extends Thread { - - // Testing commit on branch - private InputStream in = null; - private PrintStream out = null; - private String index; - - public InputStream getIn() { - return in; - } - public void setIn(InputStream in) { - this.in = in; - } - public PrintStream getOut() { - return out; - } - public void setOut(PrintStream out) { - this.out = out; - } - public String getIndex() { - return index; - } - public void setIndex(String index) { - this.index = index; - } - - public void run() { - BufferedReader stdInput = new BufferedReader(new InputStreamReader(in)); - String s; - try { - while ((s = stdInput.readLine()) != null) { - out.println("Replica " + index + ")" +s); - } - } catch(IOException ioe) { - System.out.println("----------- Exception writing replica log: " + ioe.getMessage()); - } - } -} diff --git a/test/bftsmart/demo/bftmapjunit/ConsoleTest.java b/test/bftsmart/demo/bftmapjunit/ConsoleTest.java deleted file mode 100644 index c1236d047..000000000 --- a/test/bftsmart/demo/bftmapjunit/ConsoleTest.java +++ /dev/null @@ -1,218 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.demo.bftmapjunit; - -import static org.junit.Assert.*; - -import java.io.IOException; -import java.util.HashMap; - -import org.junit.AfterClass; -import org.junit.BeforeClass; -import org.junit.Test; - -import bftsmart.demo.bftmap.BFTMap; - -/** - * - * @author Marcel Santos - * - */ -public class ConsoleTest { - - private static Process replica0; - private static Process replica1; - private static Process replica2; - private static Process replica3; - private static String[] command = new String[5]; - - @BeforeClass - public static void startServers() { - try { - System.out.println("Starting the servers"); - command[0] = "java"; - command[1] = "-cp"; - command[2] = "bin/BFT-SMaRt.jar:lib/slf4j-api-1.5.8.jar:lib/slf4j-jdk14-1.5.8.jar:lib/netty-3.1.1.GA.jar:lib/commons-codec-1.5.jar"; - command[3] = "bftsmart.demo.bftmap.BFTMapServer"; - command[4] = "0"; - replica0 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log0 = new ConsoleLogger(); - log0.setIndex("0"); - log0.setIn(replica0.getInputStream()); - log0.setOut(System.out); - log0.start(); - - command[4] = "1"; - replica1 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log1 = new ConsoleLogger(); - log1.setIndex("1"); - log1.setIn(replica1.getInputStream()); - log1.setOut(System.out); - log1.start(); - - command[4] = "2"; - replica2 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log2 = new ConsoleLogger(); - log2.setIndex("2"); - log2.setIn(replica2.getInputStream()); - log2.setOut(System.out); - log2.start(); - - command[4] = "3"; - replica3 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log3 = new ConsoleLogger(); - log3.setIndex("3"); - log3.setIn(replica3.getInputStream()); - log3.setOut(System.out); - log3.start(); - - System.out.println("Servers started"); - - } catch(IOException ioe) { - System.out.println("Exception during BFTMapInteractiveClient test: "); - System.out.println(ioe.getMessage()); - } - } - - @AfterClass - public static void stopServers() throws IllegalArgumentException, IllegalAccessException, NoSuchFieldException, IOException { - System.out.println("Stopping servers"); - replica0.destroy(); - replica1.destroy(); - replica2.destroy(); - replica3.destroy(); - System.out.println("Servers stopped"); - } - - /** - * Test regular case where there is the creation of a table, insert of - * data and search for size of the table. - * No servers are killed nor behaves different than expected. - */ - @Test - public void testRegularCase() { - try{ - Thread.sleep(1000); - BFTMap bftMap = new BFTMap(1001); - bftMap.put("TestTable1", new HashMap()); - bftMap.putEntry("TestTable1", "key1", "value1".getBytes()); - assertEquals("Main table size should be 1", 1, bftMap.size1("TestTable1")); - - for(int i = 0; i < 100; i++) { - String key = "key" + (2+i); - String value = "value" + (2+i); - bftMap.putEntry("TestTable1", key, value.getBytes()); - } - assertEquals("Main table size should be 101", 101, bftMap.size1("TestTable1")); - - bftMap.putEntry("TestTable1", "key102", "value102".getBytes()); - assertEquals("Main table size should be 102", 102, bftMap.size1("TestTable1")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - public static void main(String[] args) { - ConsoleTest test = new ConsoleTest(); - startServers(); - test.testStopAndStartNonLeader(); - try { - stopServers(); - } catch(Exception e) { - e.printStackTrace(); - } - } - - /** - * This test insert and retrieve data. - * During this process a replica that is not the leader is killed. - * After that the replica is started back. - * During the whole process messages keep being sent, to test if - * the application works as expected. - */ - @Test - public void testStopAndStartNonLeader() { - try{ - Thread.sleep(1000); - BFTMap bftMap = new BFTMap(1001); - bftMap.put("TestTable2", new HashMap()); - - for(int i = 0; i < 65; i++) { - String key = "key" + (1+i); - String value = "value" + (2+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - - System.out.println("---------Sleep1: " + new java.util.Date()); - Thread.sleep(5000); - System.out.println("---------Wakeup1: " + new java.util.Date()); - - System.out.println("---------Killing replica 1"); - replica1.destroy(); // Killing a non-leader replica, replica1 - - System.out.println("---------Sleep2: " + new java.util.Date()); - Thread.sleep(5000); - System.out.println("---------Wakeup2: " + new java.util.Date()); - - for(int i = 0; i < 35; i++) { - String key = "key" + (66+i); - String value = "value" + (66+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - - System.out.println("---------Sleep3: " + new java.util.Date()); - Thread.sleep(5000); - System.out.println("---------Wakeup3: " + new java.util.Date()); - - command[4] = "1"; - System.out.println("---------Starting replica 1"); - replica1 = new ProcessBuilder(command).start(); // Starting replica1 back - ConsoleLogger log1 = new ConsoleLogger(); - log1.setIndex("11"); - log1.setIn(replica1.getInputStream()); - log1.setOut(System.out); - log1.start(); - - System.out.println("---------Sleep4: " + new java.util.Date()); - Thread.sleep(20000); - System.out.println("---------Wakeup4: " + new java.util.Date()); - - for(int i = 0; i < 35; i++) { - String key = "key" + (101+i); - String value = "value" + (101+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - - System.out.println("---------Sleep2: " + new java.util.Date()); - Thread.sleep(20000); - System.out.println("---------Wakeup2: " + new java.util.Date()); - - replica2.destroy(); // Killing another non-leader replica, replica3 - for(int i = 0; i < 35; i++) { - String key = "key" + (136+i); - String value = "value" + (136+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - System.out.println("----------Table size:" + bftMap.size1("TestTable2")); - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } catch(IOException ioe) { - System.out.println("Exception when starting replica 2: " + ioe.getMessage()); - } - } - -} diff --git a/test/bftsmart/demo/bftmapjunit/KVClientTest.java b/test/bftsmart/demo/bftmapjunit/KVClientTest.java deleted file mode 100644 index 399aac90d..000000000 --- a/test/bftsmart/demo/bftmapjunit/KVClientTest.java +++ /dev/null @@ -1,443 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.demo.bftmapjunit; - -import static org.junit.Assert.*; - -import java.io.IOException; -import java.util.HashMap; - -import org.junit.AfterClass; -import org.junit.BeforeClass; -import org.junit.Test; - -import bftsmart.demo.bftmap.BFTMap; - -/** - * - * @author Marcel Santos - * - */ -public class KVClientTest { - - private static Process replica0; - private static Process replica1; - private static Process replica2; - private static Process replica3; - private static String[] command = new String[5]; - - @BeforeClass - public static void startServers() { - try { - System.out.println("Starting the servers"); - command[0] = "java"; - command[1] = "-cp"; - command[2] = "bin/BFT-SMaRt.jar:lib/slf4j-api-1.5.8.jar:lib/slf4j-jdk14-1.5.8.jar:lib/netty-3.1.1.GA.jar:lib/commons-codec-1.5.jar"; - command[3] = "bftsmart.demo.bftmap.BFTMapServer"; - command[4] = "0"; - - replica0 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log0 = new ConsoleLogger();; - log0.setIn(replica0.getInputStream()); - log0.setOut(System.out); - log0.setIndex("0"); - log0.start(); - Thread.sleep(2000); - command[4] = "1"; - replica1 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log1 = new ConsoleLogger();; - log1.setIn(replica1.getInputStream()); - log1.setOut(System.out); - log1.setIndex("1"); - log1.start(); - Thread.sleep(2000); - command[4] = "2"; - replica2 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log2 = new ConsoleLogger();; - log2.setIn(replica2.getInputStream()); - log2.setOut(System.out); - log2.setIndex("2"); - log2.start(); - Thread.sleep(2000); - command[4] = "3"; - replica3 = new ProcessBuilder(command).redirectErrorStream(true).start(); - ConsoleLogger log3 = new ConsoleLogger();; - log3.setIn(replica3.getInputStream()); - log3.setOut(System.out); - log3.setIndex("3"); - log3.start(); - System.out.println("Servers started"); - - } catch(IOException ioe) { - System.out.println("Exception during BFTMapInteractiveClient test: "); - System.out.println(ioe.getMessage()); - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - @AfterClass - public static void stopServers() throws IllegalArgumentException, IllegalAccessException, NoSuchFieldException, IOException { - System.out.println("Stopping servers"); - replica0.destroy(); - replica1.destroy(); - replica2.destroy(); - replica3.destroy(); - System.out.println("Servers stopped"); - } - - /** - * Test regular case where there is the creation of a table, insert of - * data and search for size of the table. - * No servers are killed nor behaves different than expected. - */ - @Test - public void testRegularCase() { - try{ - Thread.sleep(1000); - BFTMap bftMap = new BFTMap(1001); - bftMap.put("TestTable1", new HashMap()); - bftMap.putEntry("TestTable1", "key1", "value1".getBytes()); - assertEquals("Main table size should be 1", 1, bftMap.size1("TestTable1")); - - for(int i = 0; i < 100; i++) { - String key = "key" + (2+i); - String value = "value" + (2+i); - bftMap.putEntry("TestTable1", key, value.getBytes()); - } - assertEquals("Main table size should be 101", 101, bftMap.size1("TestTable1")); - - bftMap.putEntry("TestTable1", "key102", "value102".getBytes()); - assertEquals("Main table size should be 102", 102, bftMap.size1("TestTable1")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - /** - * - Crate table; - * - Insert data; - * - Kills a replica that is not a leader; - * - Insert data; - * - Verify if the size of the table is correct. - */ - @Test - public void testStopNonLeader() { - try{ - Thread.sleep(1000); - BFTMap bftMap = new BFTMap(1001); - bftMap.put("TestTable2", new HashMap()); - bftMap.putEntry("TestTable2", "key1", "value1".getBytes()); - assertEquals("Main table size should be 1", 1, bftMap.size1("TestTable2")); - - for(int i = 0; i < 200; i++) { - String key = "key" + (2+i); - String value = "value" + (2+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - assertEquals("Main table size should be 201", 201, bftMap.size1("TestTable2")); - - replica2.destroy(); // Killing a non-leader replica, replica2 - for(int i = 0; i < 200; i++) { - String key = "key" + (202+i); - String value = "value" + (202+i); - bftMap.putEntry("TestTable2", key, value.getBytes()); - } - assertEquals("Main table size should be 401", 401, bftMap.size1("TestTable2")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - /** - * This test insert and retrieve data. - * During this process a replica that is not the leader is killed. - * After that the replica is started back. - * During the whole process messages keep being sent, to test if - * the application works as expected. - */ - @Test - public void testStopAndStartNonLeader() { - try{ - Thread.sleep(5000); - BFTMap bftMap = new BFTMap(1001); - Thread.sleep(1000); - bftMap.put("TestTable3", new HashMap()); - - for(int i = 0; i < 65; i++) { - String key = "key" + (1+i); - String value = "value" + (1+i); - System.out.println(bftMap.putEntry("TestTable3", key, value.getBytes())); - } - assertEquals("Main table size should be 65", 65, bftMap.size1("TestTable3")); - - replica1.destroy(); // Killing a non-leader replica, replica2 - for(int i = 0; i < 35; i++) { - String key = "key" + (66+i); - String value = "value" + (66+i); - System.out.println(bftMap.putEntry("TestTable3", key, value.getBytes())); - } - assertEquals("Main table size should be 100", 100, bftMap.size1("TestTable3")); - - command[4] = "1"; - replica1 = new ProcessBuilder(command).redirectErrorStream(true).start(); // Starting replica2 back - ConsoleLogger log1 = new ConsoleLogger();; - log1.setIn(replica1.getInputStream()); - log1.setOut(System.out); - log1.setIndex("11"); - log1.start(); - - System.out.println("---------Sleep1: " + new java.util.Date()); - Thread.sleep(20000); - System.out.println("---------Wakeup1: " + new java.util.Date()); - - for(int i = 0; i < 35; i++) { - String key = "key" + (101+i); - String value = "value" + (101+i); - System.out.println(bftMap.putEntry("TestTable3", key, value.getBytes())); - } - assertEquals("Main table size should be 135", 135, bftMap.size1("TestTable3")); - - System.out.println("---------Sleep2: " + new java.util.Date()); - Thread.sleep(10000); - System.out.println("---------Wakeup2: " + new java.util.Date()); - - replica2.destroy(); // Killing another non-leader replica, replica3 - - for(int i = 0; i < 35; i++) { - String key = "key" + (136+i); - String value = "value" + (136+i); - System.out.println(bftMap.putEntry("TestTable3", key, value.getBytes())); - } - assertEquals("Main table size should be 170", 170, bftMap.size1("TestTable3")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } catch(IOException ioe) { - System.out.println("Exception when starting replica 2: " + ioe.getMessage()); - } - } - - /** - * - Crate table; - * - Insert data; - * - Kills the leader replica; - * - Insert data; - * - Verify if the size of the table is correct. - */ - @Test - public void testStopLeader() { - try{ - Thread.sleep(1000); - BFTMap bftMap = new BFTMap(1001); - Thread.sleep(1000); - bftMap.put("TestTable4", new HashMap()); - bftMap.putEntry("TestTable4", "key1", "value1".getBytes()); - assertEquals("Main table size should be 1", 1, bftMap.size1("TestTable4")); - - for(int i = 0; i < 200; i++) { - String key = "key" + (2+i); - String value = "value" + (2+i); - bftMap.putEntry("TestTable4", key, value.getBytes()); - } - assertEquals("Main table size should be 201", 201, bftMap.size1("TestTable4")); - - replica0.destroy(); // Killing the leader, replica 0 - - for(int i = 0; i < 200; i++) { - String key = "key" + (202+i); - String value = "value" + (202+i); - bftMap.putEntry("TestTable4", key, value.getBytes()); - } - assertEquals("Main table size should be 401", 401, bftMap.size1("TestTable4")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } - } - - - /** - * - Crate table; - * - Insert data; - * - Kills the leader replica; - * - Insert data; - * - Verify if the size of the table is correct; - * - Start the replica again; - * - Insert data; - * - Verify if the size of the table is correct; - * - Kills a second leader; - * - Insert data; - * - Verify if the size of the table is correct. - */ - @Test - public void testStopLeaders() { - try{ - Thread.sleep(1000); - BFTMap bftMap = new BFTMap(1001); - Thread.sleep(1000); - bftMap.put("TestTable5", new HashMap()); - - for(int i = 0; i < 130; i++) { - String key = "key" + (1+i); - String value = "value" + (1+i); - bftMap.putEntry("TestTable5", key, value.getBytes()); - } - assertEquals("Main table size should be 130", 130, bftMap.size1("TestTable5")); - - replica0.destroy(); // Killing the leader, replica 0 - - for(int i = 0; i < 60; i++) { - String key = "key" + (131+i); - String value = "value" + (131+i); - bftMap.putEntry("TestTable5", key, value.getBytes()); - } - assertEquals("Main table size should be 190", 190, bftMap.size1("TestTable5")); - - command[4] = "0"; - replica0 = new ProcessBuilder(command).redirectErrorStream(true).start(); // Starting replica0 back - ConsoleLogger log0 = new ConsoleLogger();; - log0.setIn(replica0.getInputStream()); - log0.setOut(System.out); - log0.setIndex("01"); - log0.start(); - System.out.println("---------Sleep1: " + new java.util.Date()); - Thread.sleep(20000); - System.out.println("---------Wakeup1: " + new java.util.Date()); - - for(int i = 0; i < 20; i++) { - String key = "key" + (191+i); - String value = "value" + (191+i); - System.out.println(bftMap.putEntry("TestTable5", key, value.getBytes())); - } - assertEquals("Main table size should be 210", 210, bftMap.size1("TestTable5")); - - System.out.println("---------Sleep2: " + new java.util.Date()); - Thread.sleep(10000); - System.out.println("---------Wakeup2: " + new java.util.Date()); - - replica1.destroy(); // Killing another leader replica, replica1 - - for(int i = 0; i < 60; i++) { - String key = "key" + (211+i); - String value = "value" + (211+i); - System.out.println(bftMap.putEntry("TestTable5", key, value.getBytes())); - } - assertEquals("Main table size should be 270", 270, bftMap.size1("TestTable5")); - - command[4] = "1"; - replica1 = new ProcessBuilder(command).redirectErrorStream(true).start(); // Starting replica0 back - ConsoleLogger log1 = new ConsoleLogger();; - log1.setIn(replica1.getInputStream()); - log1.setOut(System.out); - log1.setIndex("11"); - log1.start(); - System.out.println("---------Sleep3: " + new java.util.Date()); - Thread.sleep(20000); - System.out.println("---------Wakeup3: " + new java.util.Date()); - - for(int i = 0; i < 20; i++) { - String key = "key" + (271+i); - String value = "value" + (271+i); - System.out.println(bftMap.putEntry("TestTable5", key, value.getBytes())); - } - assertEquals("Main table size should be 290", 290, bftMap.size1("TestTable5")); - - System.out.println("---------Sleep4: " + new java.util.Date()); - Thread.sleep(10000); - System.out.println("---------Wakeup4: " + new java.util.Date()); - - replica2.destroy(); // Killing another leader replica, replica1 - - for(int i = 0; i < 40; i++) { - String key = "key" + (291+i); - String value = "value" + (291+i); - System.out.println(bftMap.putEntry("TestTable5", key, value.getBytes())); - } - assertEquals("Main table size should be 330", 330, bftMap.size1("TestTable5")); - - command[4] = "2"; - replica2 = new ProcessBuilder(command).redirectErrorStream(true).start(); // Starting replica0 back - ConsoleLogger log2 = new ConsoleLogger();; - log2.setIn(replica2.getInputStream()); - log2.setOut(System.out); - log2.setIndex("21"); - log2.start(); - System.out.println("---------Sleep5: " + new java.util.Date()); - Thread.sleep(20000); - System.out.println("---------Wakeup5: " + new java.util.Date()); - - for(int i = 0; i < 40; i++) { - String key = "key" + (331+i); - String value = "value" + (331+i); - System.out.println(bftMap.putEntry("TestTable5", key, value.getBytes())); - } - assertEquals("Main table size should be 370", 370, bftMap.size1("TestTable5")); - - System.out.println("---------Sleep6: " + new java.util.Date()); - Thread.sleep(10000); - System.out.println("---------Wakeup6: " + new java.util.Date()); - - replica3.destroy(); // Killing another leader replica, replica1 - - for(int i = 0; i < 40; i++) { - String key = "key" + (371+i); - String value = "value" + (371+i); - System.out.println(bftMap.putEntry("TestTable5", key, value.getBytes())); - } - assertEquals("Main table size should be 410", 410, bftMap.size1("TestTable5")); - - command[4] = "3"; - replica3 = new ProcessBuilder(command).redirectErrorStream(true).start(); // Starting replica0 back - ConsoleLogger log3 = new ConsoleLogger();; - log3.setIn(replica3.getInputStream()); - log3.setOut(System.out); - log3.setIndex("31"); - log3.start(); - System.out.println("---------Sleep7: " + new java.util.Date()); - Thread.sleep(20000); - System.out.println("---------Wakeup7: " + new java.util.Date()); - - for(int i = 0; i < 20; i++) { - String key = "key" + (411+i); - String value = "value" + (411+i); - System.out.println(bftMap.putEntry("TestTable5", key, value.getBytes())); - } - assertEquals("Main table size should be 430", 430, bftMap.size1("TestTable5")); - - System.out.println("---------Sleep8: " + new java.util.Date()); - Thread.sleep(10000); - System.out.println("---------Wakeup8: " + new java.util.Date()); - - replica0.destroy(); // Killing another leader replica, replica1 - - for(int i = 0; i < 40; i++) { - String key = "key" + (431+i); - String value = "value" + (431+i); - System.out.println(bftMap.putEntry("TestTable5", key, value.getBytes())); - } - assertEquals("Main table size should be 470", 470, bftMap.size1("TestTable5")); - - } catch(InterruptedException ie) { - System.out.println("Exception during Thread sleep: " + ie.getMessage()); - } catch(IOException ioe) { - System.out.println("Exception when starting replica 2: " + ioe.getMessage()); - } - } - -} diff --git a/test/bftsmart/demo/bftmapjunit/LogWriter.java b/test/bftsmart/demo/bftmapjunit/LogWriter.java deleted file mode 100644 index 02205bad0..000000000 --- a/test/bftsmart/demo/bftmapjunit/LogWriter.java +++ /dev/null @@ -1,70 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.demo.bftmapjunit; - -import java.io.BufferedReader; -import java.io.FileWriter; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.io.PrintStream; -import java.io.PrintWriter; - -/** - * - * @author Marcel Santos - * - */ -public class LogWriter extends Thread { - - private InputStream in = null; - private PrintStream out = null; - private int index; - - public InputStream getIn() { - return in; - } - public void setIn(InputStream in) { - this.in = in; - } - public PrintStream getOut() { - return out; - } - public void setOut(PrintStream out) { - this.out = out; - } - public int getIndex() { - return index; - } - public void setIndex(int index) { - this.index = index; - } - - public void run() { - BufferedReader stdInput = new BufferedReader(new InputStreamReader(in)); - String s; - try { - PrintWriter pw = new PrintWriter(new FileWriter("ServerLog-" + index + ".debug")); - while ((s = stdInput.readLine()) != null) { -// out.println(s); - pw.println(s); - } - pw.close(); - } catch(IOException ioe) { - System.out.println("----------- Exception writing replica log: " + ioe.getMessage()); - } - } -} \ No newline at end of file diff --git a/test/bftsmart/statemanagement/CSTRequestF1Test.java b/test/bftsmart/statemanagement/CSTRequestF1Test.java deleted file mode 100644 index 7727f2439..000000000 --- a/test/bftsmart/statemanagement/CSTRequestF1Test.java +++ /dev/null @@ -1,172 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.statemanagement; - -import static org.junit.Assert.*; - -import org.junit.Test; - -import bftsmart.statemanagement.strategy.durability.CSTRequestF1; - -/** - * - * @author Marcel Santos - * - */ -public class CSTRequestF1Test { - - @Test - public void testDefineReplicas() { - - // replica 3 was the last to take the checkpoint - CSTRequestF1 request = new CSTRequestF1(11700); - int[] otherProcesses = {1,2,3}; - request.defineReplicas(otherProcesses, 10000, 0); - assertEquals("Replica 2 should send the checkpoint", 2, request.getCheckpointReplica()); - assertEquals("Replica 1 should send the lower half of the log", 1, request.getLogLower()); - assertEquals("Replica 3 should send the upper half of the log", 3, request.getLogUpper()); - assertEquals("Upper log size should be 1701", 1701, request.getLogUpperSize()); - - request = new CSTRequestF1(11700); - otherProcesses[0] = 0; // 0,2,3 - request.defineReplicas(otherProcesses, 10000, 1); - assertEquals("Replica 2 should send the checkpoint", 2, request.getCheckpointReplica()); - assertEquals("Replica 0 should send the lower half of the log", 0, request.getLogLower()); - assertEquals("Replica 3 should send the upper half of the log", 3, request.getLogUpper()); - assertEquals("Upper log size should be 1701", 1701, request.getLogUpperSize()); - - request = new CSTRequestF1(11700); - otherProcesses[1] = 1; // 0,1,3 - request.defineReplicas(otherProcesses, 10000, 2); - assertEquals("Replica 1 should send the checkpoint", 1, request.getCheckpointReplica()); - assertEquals("Replica 0 should send the lower half of the log", 0, request.getLogLower()); - assertEquals("Replica 3 should send the upper half of the log", 3, request.getLogUpper()); - assertEquals("Upper log size should be 1701", 1701, request.getLogUpperSize()); - - request = new CSTRequestF1(11700); - otherProcesses[2] = 2; // 0,1,2 - request.defineReplicas(otherProcesses, 10000, 3); - assertEquals("Replica 1 should send the checkpoint", 1, request.getCheckpointReplica()); - assertEquals("Replica 0 should send the lower half of the log", 0, request.getLogLower()); - assertEquals("Replica 2 should send the upper half of the log", 2, request.getLogUpper()); - assertEquals("Upper log size should be 1701", 1701, request.getLogUpperSize()); - - // replica 0 was the last to take the checkpoint - request = new CSTRequestF1(13300); - otherProcesses[0] = 1; // 1,2,3 - otherProcesses[1] = 2; - otherProcesses[2] = 3; - request.defineReplicas(otherProcesses, 10000, 0); - assertEquals("Replica 2 should send the checkpoint", 2, request.getCheckpointReplica()); - assertEquals("Replica 1 should send the lower half of the log", 1, request.getLogLower()); - assertEquals("Replica 3 should send the upper half of the log", 3, request.getLogUpper()); - assertEquals("Upper log size should be 801", 801, request.getLogUpperSize()); - - request = new CSTRequestF1(13300); - otherProcesses[0] = 0; // 0,2,3 - request.defineReplicas(otherProcesses, 10000, 1); - assertEquals("Replica 3 should send the checkpoint", 3, request.getCheckpointReplica()); - assertEquals("Replica 2 should send the lower half of the log", 2, request.getLogLower()); - assertEquals("Replica 0 should send the upper half of the log", 0, request.getLogUpper()); - assertEquals("Upper log size should be 801", 801, request.getLogUpperSize()); - - request = new CSTRequestF1(13300); - otherProcesses[1] = 1; // 0,1,3 - request.defineReplicas(otherProcesses, 10000, 2); - assertEquals("Replica 3 should send the checkpoint", 3, request.getCheckpointReplica()); - assertEquals("Replica 1 should send the lower half of the log", 1, request.getLogLower()); - assertEquals("Replica 0 should send the upper half of the log", 0, request.getLogUpper()); - assertEquals("Upper log size should be 801", 801, request.getLogUpperSize()); - - request = new CSTRequestF1(13300); - otherProcesses[2] = 2; // 0,1,2 - request.defineReplicas(otherProcesses, 10000, 3); - assertEquals("Replica 2 should send the checkpoint", 2, request.getCheckpointReplica()); - assertEquals("Replica 1 should send the lower half of the log", 1, request.getLogLower()); - assertEquals("Replica 0 should send the upper half of the log", 0, request.getLogUpper()); - assertEquals("Upper log size should be 801", 801, request.getLogUpperSize()); - - // replica 1 was the last to take the checkpoint - request = new CSTRequestF1(16200); - otherProcesses[0] = 1; // 1,2,3 - otherProcesses[1] = 2; - otherProcesses[2] = 3; - request.defineReplicas(otherProcesses, 10000, 0); - assertEquals("Replica 3 should send the checkpoint", 3, request.getCheckpointReplica()); - assertEquals("Replica 2 should send the lower half of the log", 2, request.getLogLower()); - assertEquals("Replica 1 should send the upper half of the log", 1, request.getLogUpper()); - assertEquals("Upper log size should be 1201", 1201, request.getLogUpperSize()); - - request = new CSTRequestF1(16200); - otherProcesses[0] = 0; // 0,2,3 - request.defineReplicas(otherProcesses, 10000, 1); - assertEquals("Replica 3 should send the checkpoint", 3, request.getCheckpointReplica()); - assertEquals("Replica 2 should send the lower half of the log", 2, request.getLogLower()); - assertEquals("Replica 0 should send the upper half of the log", 0, request.getLogUpper()); - assertEquals("Upper log size should be 1201", 1201, request.getLogUpperSize()); - - request = new CSTRequestF1(16200); - otherProcesses[1] = 1; // 0,1,3 - request.defineReplicas(otherProcesses, 10000, 2); - assertEquals("Replica 0 should send the checkpoint", 0, request.getCheckpointReplica()); - assertEquals("Replica 3 should send the lower half of the log", 3, request.getLogLower()); - assertEquals("Replica 1 should send the upper half of the log", 1, request.getLogUpper()); - assertEquals("Upper log size should be 1201", 1201, request.getLogUpperSize()); - - request = new CSTRequestF1(16200); - otherProcesses[2] = 2; // 0,1,2 - request.defineReplicas(otherProcesses, 10000, 3); - assertEquals("Replica 0 should send the checkpoint", 0, request.getCheckpointReplica()); - assertEquals("Replica 2 should send the lower half of the log", 2, request.getLogLower()); - assertEquals("Replica 1 should send the upper half of the log", 1, request.getLogUpper()); - assertEquals("Upper log size should be 1201", 1201, request.getLogUpperSize()); - - // replica 2 was the last to take the checkpoint - request = new CSTRequestF1(17700); - otherProcesses[0] = 1; // 1,2,3 - otherProcesses[1] = 2; - otherProcesses[2] = 3; - request.defineReplicas(otherProcesses, 10000, 0); - assertEquals("Replica 1 should send the checkpoint", 1, request.getCheckpointReplica()); - assertEquals("Replica 3 should send the lower half of the log", 3, request.getLogLower()); - assertEquals("Replica 2 should send the upper half of the log", 2, request.getLogUpper()); - assertEquals("Upper log size should be 201", 201, request.getLogUpperSize()); - - request = new CSTRequestF1(17700); - otherProcesses[0] = 0; // 0,2,3 - request.defineReplicas(otherProcesses, 10000, 1); - assertEquals("Replica 0 should send the checkpoint", 0, request.getCheckpointReplica()); - assertEquals("Replica 3 should send the lower half of the log", 3, request.getLogLower()); - assertEquals("Replica 2 should send the upper half of the log", 2, request.getLogUpper()); - assertEquals("Upper log size should be 201", 201, request.getLogUpperSize()); - - request = new CSTRequestF1(17700); - otherProcesses[1] = 1; // 0,1,3 - request.defineReplicas(otherProcesses, 10000, 2); - assertEquals("Replica 0 should send the checkpoint", 0, request.getCheckpointReplica()); - assertEquals("Replica 3 should send the lower half of the log", 3, request.getLogLower()); - assertEquals("Replica 1 should send the upper half of the log", 1, request.getLogUpper()); - assertEquals("Upper log size should be 201", 201, request.getLogUpperSize()); - - request = new CSTRequestF1(17700); - otherProcesses[2] = 2; // 0,1,2 - request.defineReplicas(otherProcesses, 10000, 3); - assertEquals("Replica 1 should send the checkpoint", 1, request.getCheckpointReplica()); - assertEquals("Replica 0 should send the lower half of the log", 0, request.getLogLower()); - assertEquals("Replica 2 should send the upper half of the log", 2, request.getLogUpper()); - assertEquals("Upper log size should be 201", 201, request.getLogUpperSize()); - } -} diff --git a/test/bftsmart/tom/ServiceProxyTest.java b/test/bftsmart/tom/ServiceProxyTest.java deleted file mode 100644 index 16a632de3..000000000 --- a/test/bftsmart/tom/ServiceProxyTest.java +++ /dev/null @@ -1,180 +0,0 @@ -/** -Copyright (c) 2007-2013 Alysson Bessani, Eduardo Alchieri, Paulo Sousa, and the authors indicated in the @author tags - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ -package bftsmart.tom; - -import java.lang.reflect.Field; - -import junit.framework.Assert; - -import org.junit.BeforeClass; -import org.junit.Test; - -import bftsmart.tom.core.messages.TOMMessage; -import bftsmart.tom.core.messages.TOMMessageType; - -/** - * - * @author Marcel Santos - * - */ -public class ServiceProxyTest { - - private static ServiceProxy proxy; - - @BeforeClass - public static void setup(){ - proxy = new ServiceProxy(1001); - } - - @Test - public void testReplyReceived() { - // Just to setup some variables - - Field response; - Field receivedReplies; - Field reqId; - Field replyQuorum; - Field replies; - Field requestType; - - try { - response = ServiceProxy.class.getDeclaredField("response"); - response.setAccessible(true); - receivedReplies = ServiceProxy.class.getDeclaredField("receivedReplies"); - receivedReplies.setAccessible(true); - reqId = ServiceProxy.class.getDeclaredField("reqId"); - reqId.setAccessible(true); - replyQuorum = ServiceProxy.class.getDeclaredField("replyQuorum"); - replyQuorum.setAccessible(true); - replies = ServiceProxy.class.getDeclaredField("replies"); - replies.setAccessible(true); - requestType = ServiceProxy.class.getDeclaredField("requestType"); - requestType.setAccessible(true); - - try { - // Test if a the method decides correctly when three correct - // replies are received in sequence. The request are ordered - TOMMessage[] initReplies = new TOMMessage[4]; - receivedReplies.set(proxy, 0); - reqId.setInt(proxy, 1); - replyQuorum.setInt(proxy, 3); - replies.set(proxy, initReplies); - requestType.set(proxy, TOMMessageType.ORDERED_REQUEST); - response.set(proxy, null); - - TOMMessage msg = new TOMMessage(0, 1, 1, "response1".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(1, 1, 1, "response1".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(2, 1, 1, "response1".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - TOMMessage reply = (TOMMessage)response.get(proxy); - - Assert.assertEquals("response1", new String(reply.getContent())); - - // Test if a the method decides correctly when the replies are - // correct, correct, wrong and correct, in that order. The requests - // are ordered - initReplies = new TOMMessage[4]; - receivedReplies.set(proxy, 0); - reqId.setInt(proxy, 1); - replyQuorum.setInt(proxy, 3); - replies.set(proxy, initReplies); - requestType.set(proxy, TOMMessageType.ORDERED_REQUEST); - response.set(proxy, null); - - msg = new TOMMessage(0, 1, 1, "response1".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(1, 1, 1, "response1".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(2, 1, 1, "response2".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(3, 1, 1, "response1".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - reply = (TOMMessage)response.get(proxy); - - Assert.assertEquals("response1", new String(reply.getContent())); - - // Negative test, to verify if the method doesn't decide when the values - // doesn't match as a reply quorum. The values are 2 response1 and 2 response2 - initReplies = new TOMMessage[4]; - receivedReplies.set(proxy, 0); - reqId.setInt(proxy, 1); - replyQuorum.setInt(proxy, 3); - replies.set(proxy, initReplies); - requestType.set(proxy, TOMMessageType.ORDERED_REQUEST); - response.set(proxy, null); - - msg = new TOMMessage(0, 1, 1, "response1".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(1, 1, 1, "response1".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(2, 1, 1, "response2".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(3, 1, 1, "response2".getBytes(), 0, TOMMessageType.ORDERED_REQUEST); - proxy.replyReceived(msg); - reply = (TOMMessage)response.get(proxy); - - Assert.assertNull(reply); - Assert.assertEquals(-1, reqId.get(proxy)); - - // Test replyReceived for correct readonly messages - initReplies = new TOMMessage[4]; - receivedReplies.set(proxy, 0); - reqId.setInt(proxy, 1); - replies.set(proxy, initReplies); - requestType.set(proxy, TOMMessageType.UNORDERED_REQUEST); - response.set(proxy, null); - - msg = new TOMMessage(0, 1, 1, "response1".getBytes(), 0, TOMMessageType.UNORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(1, 1, 1, "response1".getBytes(), 0, TOMMessageType.UNORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(2, 1, 1, "response1".getBytes(), 0, TOMMessageType.UNORDERED_REQUEST); - proxy.replyReceived(msg); - - reply = (TOMMessage)response.get(proxy); - Assert.assertEquals("response1", new String(reply.getContent())); - - // Test if the method fails for the first diverging readonly reply - receivedReplies.set(proxy, 0); - reqId.setInt(proxy, 1); - replies.set(proxy, initReplies); - requestType.set(proxy, TOMMessageType.UNORDERED_REQUEST); - response.set(proxy, null); - - msg = new TOMMessage(0, 1, 1, "response1".getBytes(), 0, TOMMessageType.UNORDERED_REQUEST); - proxy.replyReceived(msg); - msg = new TOMMessage(1, 1, 1, "response2".getBytes(), 0, TOMMessageType.UNORDERED_REQUEST); - proxy.replyReceived(msg); - - reply = (TOMMessage)response.get(proxy); - Assert.assertEquals(-1, reqId.get(proxy)); - - } catch (IllegalArgumentException | IllegalAccessException e) { - - e.printStackTrace(); - } - } catch (NoSuchFieldException e) { - e.printStackTrace(); - } catch (SecurityException e) { - // TODO Auto-generated catch block - e.printStackTrace(); - } - } - - -}