Example not clear

[sarfarazengglb]

The example provided are NOT so clear for consumer.How to implement in client server passion or point to point communication.There is no sample for communication with Port and IP nor there is no example to know how to encrypt.Can you create community forum where we can share issues and communicate along with providing full documentation

[andy-trimble]

The port and IP address for both consumers and producers are automatically discovered. There is no need to specify them, but you can if you need to. Please refer to the Wiki Page on configuration.

I agree that there needs to be an example on encryption.

Please continue to use the issue reporting and Wiki as a community forum.

[sarfarazengglb]

I have fixed the example and did a test run.It is giving 148 message/sec.When i am sending the message there is lot of latency more than 102 sec in single message.I submitted the message at 2015-10-09 02:53:14.331 and delivered at 2015-10-09 02:54:56.475.Along with that there is a pause of few second where it prints decode message.Can you please advice how to reduce.I need to reduce this latency to less than 5 sec in each message,

02:54:56.475 [ringBuffer-5] INFO i.bigio.examples.helloworld.Consumer - As an anonymous inner class: MsgID 326572Msg Time=>2015-10-09 02:53:14.331=TimeNow =>2015-10-09 02:54:56.475

[andy-trimble]

It shouldn't take anywhere near that much time. Can you post your code please? What's your OS and Java version?

[sarfarazengglb]

Hi Andy,
Please find the below details.There is one more issue which i will raise in separate thread.

My code is as follows
package io.bigio.examples.helloworld;

import io.bigio.BigIO;
import io.bigio.Initialize;
import io.bigio.Inject;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.sql.Timestamp;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.logging.Level;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
*

• @author atrimble
  */
  //@component
  public class Producer {

  private static final Logger LOG = LoggerFactory.getLogger(Producer.class);

  @Inject
  private BigIO bigio;

  @initialize
  public void init() {
  new Thread() {
  @OverRide
  public void run() {
  int inc = 0;
  Timestamp time=new Timestamp(new Date().getTime());
  LOG.info("Start Time"+time);
  BufferedReader bufferRead = new BufferedReader(new InputStreamReader(System.in));
  try {
  String messageValue= bufferRead.readLine();
  } catch (IOException ex) {
  java.util.logging.Logger.getLogger(Producer.class.getName()).log(Level.SEVERE, null, ex);
  }
  while (inc <5000000) {
  try {
  HelloWorldMessage msg=new HelloWorldMessage();
  msg.setMessage("MsgID " + inc + "Msg Time=>"+new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSSSSS").format(new Date()));
  LOG.info("Sending message at "+new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSSSSS").format(new Date()));
  LOG.info("Msg "+msg.getMessage());
  bigio.send("Topic",msg);
  inc++;
  } catch (IOException ex) {
  LOG.error("Caught exception", ex);
  //} catch (InterruptedException ex) {
  // java.util.logging.Logger.getLogger(Producer.class.getName()).log(Level.SEVERE, null, ex);
  }
  }
  LOG.info("End Time"+new Timestamp(new Date().getTime()));
  }
  }.start();
  }

  public static void main(String[] args) {

  Producer prod = new Producer();
  prod.bigio = BigIO.bootstrap();
  prod.init();
  

//prod.bigio.
}
}

package io.bigio.examples.helloworld;

import io.bigio.BigIO;
import io.bigio.Component;
import io.bigio.Initialize;
import io.bigio.Inject;
import io.bigio.Interceptor;
import io.bigio.MessageListener;
import io.bigio.core.Envelope;
import java.sql.Timestamp;
import java.text.SimpleDateFormat;
import java.util.Date;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
*

• @author atrimble
  */
  @component
  public class Consumer {

  private static final Logger LOG = LoggerFactory.getLogger(Consumer.class);

  @Inject
  private BigIO speaker;

  @initialize
  public void init() {
  getSpeaker().addListener("Topic", (HelloWorldMessage message) -> {
  // LOG.info("As a lambda expression: " + message.getMessage());
  });

  getSpeaker().addListener("Topic", new MessageListener<HelloWorldMessage>() {
      @Override
      public void receive(HelloWorldMessage message) {
          LOG.info("As an anonymous inner class: " + message.getMessage()+"=TimeNow =>"+new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSSSSS").format(new Date()));
      }
  });
  
  getSpeaker().addInterceptor("Topic", (Envelope envelope) -> {
     // LOG.info("Interceptor as a lambda expression: " + envelope.getSenderKey() + " : " + envelope.getTopic());
      return envelope;
  });
  
  getSpeaker().addInterceptor("Topic", new Interceptor() {
      @Override
      public Envelope intercept(Envelope envelope) {
      //    LOG.info("Interceptor as an anonymous inner class: " + envelope.getSenderKey() + " : " + envelope.getTopic());
          return envelope;
      }
  });
  

  }

  /**

  • @return the speaker
    */
    public BigIO getSpeaker() {
    return speaker;
    }

  /**

  • @param speaker the speaker to set
    */
    public void setSpeaker(BigIO speaker) {
    this.speaker = speaker;
    }

    public static void main(String[] args) {

    Consumer consumer = new Consumer();
    consumer.setSpeaker(BigIO.bootstrap());
    consumer.init();

//prod.bigio.
}
}

package io.bigio.examples.helloworld;

import io.bigio.Message;
import io.bigio.core.codec.BigIOMessage;
import java.io.Serializable;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.text.SimpleDateFormat;
import java.util.Date;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
*

• @author atrimble
  */
  @message
  public class HelloWorldMessage implements BigIOMessage,Serializable {

  private static final Logger LOG = LoggerFactory.getLogger(HelloWorldMessage.class);
  private String message;

  //private String payLoad;
  public String getMessage() {
  return message;
  }

  public void setMessage(String message) {
  this.message = message;
  }

  @OverRide
  public void bigiodecode(byte[] bytes) {
  LOG.info("decode Start" + new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSSSSS").format(new Date()));
  this.message = new String(bytes, StandardCharsets.UTF_8);
  LOG.info("decode End" + new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSSSSS").format(new Date()));
  }

  @OverRide
  public byte[] bigioencode() throws Exception {
  LOG.info("Encode Start" + new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSSSSS").format(new Date()));
  return this.message.getBytes(Charset.forName("UTF-8"));

  }
  }

[andy-trimble]

Ah, I see the problem. You should not directly implement BigIOMessage or Serializable in the HelloWorldMessage class. That capability is automatically injected by the BigIO agent. By virtue of adding the -javaagent flag to your run script, the message serialization is handled automatically.

Let me know if this resolves the latency.

[sarfarazengglb]

It is working with annotation.The rate i am getting is Producer 583 Message/sec and consumer rate of 404 message/sec.There is delay when consumer accept the message may be few millsecond which adds up with each message.I have passed the created time in message and printed current time and message in the Consumer 's receiver .How much it support by producer and consumer.

Only One Receiver is registered
bigio.addListener("MessageChannel", new MessageListener() {
@OverRide
public void receive(Msg message) {
LOG.info("Recived message[" + message.getMessage() + "] at " + new Timestamp(new Date().getTime()));
}
});

Total Message Produced : 1.4 million message in 583 Message/sec in 40 minutes
Total Message Pushed : 1.2 million message in 500 Message/sec in 40 minutes

Do advise best practice to get 150,000 messages sent and 150,000 messages received

[andy-trimble]

Are you still manually implementing bigioencode and bigiodecode?

[sarfarazengglb]

No.I am using all default setting.Nothing Manual using same code of Example

Regards,

Mohammed

On Tue, Oct 13, 2015 at 8:46 PM, Andrew Trimble notifications@github.com
wrote:

Are you still manually implementing bigioencode and bigiodecode?

—
Reply to this email directly or view it on GitHub
#32 (comment).