SIM800L stops sendind data after few times

Miha_Horvat · December 3, 2018, 8:30am

Hello!

I am using SIM800L with Arduino Uno. I send data to server over GPRS, TCP port. I have correct power supply with enough current. My problem is that SIM800L sends data a few times successfully. After that something weird happens. Response to AT commands is OK. Module also connects with server but it doesn’t send data. Sometimes does not send anything, sometimes it sends AT commands as text. In CIPSEND I only send raw data of measurements. If I reset module (remove power supply and connect back) it starts to work properly again.

Here is my code I use in function to send data to server:
myGsm.println(“AT+CIPSHUT”);
delay(1000);
printSerialData();

myGsm.println(“AT+CIPMUX=0”);
delay(2000);
printSerialData();

myGsm.println(“AT+CGATT=1”);
delay(1000);
printSerialData();

myGsm.println(“AT+CSTT=“internet”,“mobitel”,“internet””);//setting the APN,2nd parameter empty works for all networks
delay(5000);_
printSerialData();

myGsm.println();
myGsm.println(“AT+CIICR”);
delay(6000);
printSerialData();

myGsm.println(“AT+CIFSR”); //init the HTTP request
delay(2000);
printSerialData();

myGsm.println("AT+CIPSTART=“TCP”,“xxx.xxx.xxx.xxx”,“9999"”);
delay(5000);
printSerialData();
delay(5000);

myGsm.println(“AT+CIPSEND”);
delay(2000);
printSerialData();
readSD();
myGsm.write(0x1A);

delay(3000);
myGsm.println(“AT+CIPCLOSE”);
printSerialData();

myGsm.println(“AT+CIPSHUT”);
delay(1000);
printSerialData();

Does anyone know what could be the problem? I already tried CFUN = 0 and then CFUN = 1, or CFUN = 1,1. But no result. Please help.

Thanks in advance!

ParthTemkar · December 4, 2018, 6:18am

Hi,

First Of All What is the “printSerialData()” Function Doing?
what kind of data are you trying to send? as i cannot see any data being sent, your just trying to connect to the TCP server.
Have you tried using the TinyGsm Library?
If possible Avoid Using So many Lengthy Delays in your code. as you dont know if the module is acting based on the time. so read back what your module is sending to you .

Cheers!
Parth Temkar

RaviPujar · December 4, 2018, 7:10am

Agree with @ParthTemkar You should use standard libraries instead of hard coded delays.
You are getting AT command as text because after CIPSEND you are not waiting for > prompt. You are just waiting for few seconds and sending data and next command.

Miha_Horvat · December 4, 2018, 12:57pm

Hi @ParthTemkar, thanks for reply.

printSerialData() prints result of AT commands -> for debugging purposes
I store some text and numeric values on SD card, then read from SD card and send as data (on server side is TCP server which reads NetworkStream)
Not yet. Does TinyGsmLibrary supports those functions? Does it have methods/functions that I am supposed to use?
Ok.

Thanks!

Miha_Horvat · December 4, 2018, 1:02pm

@RaviPujar thanks for reply. There must be some problem in my code as it always works fine for 4 or 5 times. Working fine means that sends data to server without problem. After that by > prompt I just get empty square and sending is done. It should display data read from SD card. I will try TinyGsmLibrary. If anything possible, let me know.

Thanks!

ParthTemkar · December 4, 2018, 1:34pm

Hi @Miha_Horvat,

as far as ive used the tiny gsm library , it has the TCP Functionality in it. I have heavily used it with MQTT and it just works like a charm, as far as i have noticed it uses the TCP connection itself. so if you can tweak it for your needs then i guess it will be useful for you.

one small suggestion i will give you is try and use the Millis in this project, Keep one instance of your GSM Connection active and other instance to read your SD card.
In a nutshell what you ask the micro-controller to do is a repetitive task on a tick of a clock.
for example
on 10th Second --> Establish The TCP connection.
on 15th Second --> read the Sd Card Values
As the timing goes you repeat these tasks every 10th second and 15th second of the tick. (Hope you got it)

//The time in MilliSeconds 
#define Int1 1000
#define Int2 5000
#define Int3 10000

long lastVal1 = 0;
long lastVal2 = 0;
long lastVal3 = 0;

void loop() {
  unsigned long currentTime = millis();

  if (currentTime - lastVal1 > Int1) {
    lastVal1 = currentTime;
    Serial.print("I am First Timer Of 1 Sec | Millis:- ");
    Serial.print(lastVal1 / 1000);
    Serial.println(" Sec");
    //Task 1
  }

   if (currentTime - lastVal2 > Int2) {
    lastVal2 = currentTime;
    Serial.print("I am Second Timer Of 5 Sec | Millis:- ");
    Serial.print(lastVal2 / 1000);
    Serial.println(" Sec");
    //Task2
  }

   if (currentTime - lastVal3 > Int3) {
    lastVal3 = currentTime;
    Serial.print("I am Third Timer Of 10 Sec | Millis:- ");
    Serial.print(lastVal3 / 1000);
    Serial.println(" Sec");
    //Task 3
  }
  
}

That’s a sample code, Create your function to maybe keep TCP alive and to read sd card and put that under any time line, like under the task you like. and Do Not Put Delay In this Or It Will Slow Down The Whole Flow …

Do let me know if that Helps you in anyway. Just keep the Connection In your Setup and the Alive Function in the Main Loop.

Im sharing a screenshot of how i usually do the tasks.

Hope this solves your problem.

Cheers!
Parth Temkar

Miha_Horvat · December 4, 2018, 1:48pm

@ParthTemkar thanks for great explanation! I will give it a try! Thanks!

ParthTemkar · December 4, 2018, 2:03pm

@Miha_Horvat anytime

feel free to share if that works for you or not.

Cheers!
Parth Temkar

Miha_Horvat · December 5, 2018, 11:01am

@ParthTemkar and @RaviPujar, I figured out the problem! Issue is that I am storing data on SD card and in sendToServer method I read data from SD card. I think that problem here is with so called “multitasking”. While controller is executing sendToServer method, it can’t at the same time read from SD card. I must somehow seperate these events. I tried disabling reading from SD card and set static string which looks just like data I was reading from SD card. I works perfectly!

@ParthTemkar , as you suggested, would Millis solve my issue? I was not able yet to test it out. I will try it in the next few hours.

Thanks in advance!

ParthTemkar · December 5, 2018, 12:19pm

Hi, @Miha_Horvat

Now you know what the problem is , if your okay, feel free to share the code. and Yes!, The “Multitasking is what Does the Trick”.

and Yes!, Millis Will solve your issue, like i shared the sample code with you, simply use that, and it will fix your problem. To Give you an example, The screenshot i shared with you was of a smart ambulance project i did for some engineering Students.
The whole Project used around 5-6 sensors, 1 Gps Module, 1 Gsm Module, Lcd.

Let Me share with you how it Actually works.
From Startup.

It Initializes the GPS and it waits until it locks on the satellite. (When i say “Waits”, It actually Tries to send Data Through Uart to the Module, Not based on Delay, and it checks for incoming Data)
It initializes the GSM and waits until it connects to the network and connects to the could server. (When i say “Waits”, It actually Tries to send Data Through Uart to the Module, Not based on Delay, and it checks for incoming Data)
after initializing all the sensor and modules, it would just sleep.
when a distress button is pressed this is what it did,
==> Send SMS to Traffic Police
==> Check the connection with the sever
==> Send Blood Pressure Data to Cloud
==> Send ECG Data To Cloud
==> Send GPS Location To Cloud
==> Send Temperature To Cloud
==> Send Pulse Data To Cloud

As well Show All these things on the LCD in real Time!.

and i did not use any delay in the whole project except , in The Main void Setup(). where all the Initialization Took Place.

So like i said, you just have to train the controller to do a Job on a Time , As the time goes to other thing, The task should shift!

Hope you understood what exactly you can do with just a simple Millis or in a nutshell for any micro-controller, you use a Hardware Timer, so your CPU is always Free and has a lot of things which it can do.

Cheers!
Parth Temkar

Miha_Horvat · December 5, 2018, 12:47pm

@ParthTemkar, thanks a lot for answer! So far I’ve never used millis, so please be patient if I don’t understand everything as I should. Once again I post my function to send data on server and one to read data from SD card. I’d like to send data to server every 30 minutes. Reason for this is that I am building data logger which has to save power with not being connected all the time. I have some simple int counter, which whenever reaches value 30, will trigger sendToServer function. Please ignore that I use AT commands since I could not implement TinyGsm library yet. And also ignore delays. Talking about delays, should I “wrap” every AT command with millis if I use this function only once every 30 minutes?

If it isn’t too much for you, could you show me on my code how could I solve my problem?

Thanks in advance!

Here is code sendToServer:

void sendToServer()
{
serialAT.println(“AT+CIPSHUT”);
delay(1000);
printSerialData();

serialAT.println(“AT+CIPMUX=0”);
delay(2000);
printSerialData();

serialAT.println(“AT+CGATT=1”);
delay(1000);
printSerialData();

serialAT.println(“AT+CSTT=“internet”,“mobitel”,“internet””);
delay(5000);
printSerialData();

serialAT.println(“AT+CIICR”);
delay(6000);
printSerialData();

serialAT.println(“AT+CIFSR”);
delay(2000);
printSerialData();

serialAT.println("AT+CIPSTART=“TCP”,“195.210.249.171”,“9999"”);
delay(4000);
printSerialData();
delay(4000);

serialAT.println(“AT+CIPSEND”);

delay(1000);
readSD();
delay(1000);

serialAT.write(0x1A);

delay(2000);
serialAT.println(“AT+CIPCLOSE”);
printSerialData();

serialAT.println(“AT+CIPSHUT”);
delay(1000);
printSerialData();

}

void readSD()
{
serverFile=SD.open(“msg.txt”);
if(serverFile)
{
while(serverFile.available())
{
serialAT.write(serverFile.read());
    //printSerialData();
  }
  //serverFile.close();

}
SD.remove("msg.txt");
}

Miha_Horvat · December 5, 2018, 1:13pm

@ParthTemkar It may be confusing for you - data I read from SD is read directly to GSM module for sending (AT+CIPSEND --> readSD()). I store data to SD because it takes too much space for global variebles on my Arduino (ATMEGA 328P).

ParthTemkar · December 5, 2018, 1:25pm

Hi @Miha_Horvat,
Please give me some time to review this and make the code. (as i am not habitual to help people through forums, its usually one to one conversation on a call, apologies if i respond late)

Besides that, what exactly are you trying to log? and what i can see from here, you used a while to read the data from the SD card, “While Loop” Usually hangs up all the other functions which are in use, its as good as using a delay, for that i have an alternative solution but i dont know if it will work effectively. Please give me some time to test out this code what you have given to me, and do you use any initialization method in your void Setup() To see if your GSM has got connected and has a active internet connection?

Cheers!
Parth Temkar

Miha_Horvat · December 5, 2018, 1:37pm

No problem.

I am logging for example such lines:
VSB_min,12/05/2018 14:24:57,0.00
VSB_max,12/05/2018 14:24:57,1.28
VSB_avg,12/05/2018 14:24:57,0.64

This format is used for server, where is SCADA system with Historian. It needs such format. I write these lines to SD card, to some file. readSD is actually not in while loop, it’s in while only for writing to SD purposes (while file available).

I laso managed using millis in loop. I am just not sure how to use millis for reading from SD. For example: I read values from sensor every second. Every 30 seconds I try to send data to server. So i start sendToServer funtcion. Problem is just how to correctly read from SD at the right time - this is when AT+CIPSEND commands is written to GSM module. Do you have any ideas?

My setup is this:

serialAT.begin(9600);
 modem.begin();  
 Serial.begin(9600);  
 SD.begin(10);
 rtc.begin();

Thanks in advance!

ParthTemkar · December 5, 2018, 2:05pm

alright, give me 10mins to look into this and send you a layout of the code.
meanwhile you can inbox me or probably if your okay share the full code.

ParthTemkar · December 5, 2018, 2:36pm

Hi, @Miha_Horvat

i looked around with what you are trying to do and came up with some options which you need to select and which suits you the most. i haven’t really played around a lot with the TCP connection so i cannot make any major suggestions on this one, so i had to look on internet for the same but couldn’t find a solution to it. so i need @RaviPujar expertise here only then probably my solutions will work.

Option 1:
@RaviPujar , So basically, is it possible to make the TCP connection Only once, and then maintain it in some way and just force the Data, Thats the CIPSEND. through it ?
and what did you mean by Creating Transparent TCP Connection In This Post?
If Yes!
Then
@Miha_Horvat You can just initialize your TCP connection and keep it active in the setup itself, and then keep on sending your data over the 30mins time frame, with the use of the millis.

Option 2:
What i saw in many of the posts, people have used a lot of delay to establish the TCP connection, @RaviPujar is i really necessary ? can’t we eliminate the delays and use a read back function?
If Yes!
Then @Miha_Horvat
What you can do is , on very 30Min Tick, you can initialize your whole Gsm and then send the Data through!, But here is also a catch, That is send the data Part by Part, Read your File Size, store some in your Buffer, clear that size and update that buffer to the CIPSEND. that’s something i would recommend.

Option3
Is it really necessary to use TCP? Can’t you use MQTT?
If Yes,
Then its just a piece of cake to read the data from the SD card and Send to The server, But again, use a Non-Blocking Mqtt Loop, and In your Main Loop, Use all the Functions, on the Click of the Clock, like what ive been saying so far .

I hope i was able to make things clear over here, im sorry i am not really great at writing exactly what is going in my mind, Apologies.

Do Let me know what suits you.

Miha_Horvat · December 5, 2018, 4:42pm

@ParthTemkar, thanks for your time and answers! As this data logger is project for my diploma (degree) I will not change to MQTT because I already wrote a lot of contents and I got everything working on my server side. I solved my problem simply without accessing SD card for sending data to server. I created arrays for each of values I am sending (min, max, avg). I store values in arrays and in CIPSEND I go across whole array, send to server, clear arrays and so on. It works OK. I will take a look at MQTT and other solutions you suggested. This is my first project. And definitely not last! I will test out MQTT for my next projects as it’s great for telemetry, which we use a lot at work. Thanks a lot!

ParthTemkar · December 5, 2018, 5:07pm

Well, your most welcome, but we still haven’t reached to the conclusion. I thought it’s more like some major project your doing and not just a diploma or a degree project.

What I will suggest you to do is , because it’s a college project, you don’t have Time to showcase the whole project where you said, you have to take the data every 30mins no one is going to do that, so usually I recommend my students to just do some basic bug fixing when such problem comes in.

What I will recommend you to do is only store let’s say 512bytes of data on your SD card as soon as you save the 512Bytes of SD card , fetch that data from the SD card and then save it into a Buffer. When you see there’s a new data in the buffer , just push that on to the Gsm module. (Here of course you need to establish the tcp connection beforehand) . Meanwhile , when the 1st buffer of 512Bytes is busy sending the data , you can keep another buffer like buffer 2 , which will simultaneously, fetch data from the SD card …

Now the process is more like this ,
In setup,
==> Init the Gsm and connect to TCP
==> init your other stuff

In Loop:
==> on every 1sec Tick, get sensor data and store on SD card
==> On every 5sec Tick, check SD card if it has 512Bytes, if yes, save it in the buffer , make a boolen for the buffer 1 as true

==> If the buffer 1 Boolean is true and tick is on 10th second, send the buffer 1 data.
Else if , the buffer 1 != True and buffer 2 is true, send the buffer 2 data.
Set the buffers Boolean to the opposite of what it was.

And keep on doing this! .

So what I just told you now, is a procedure or I can say an idea which you can follow, all you have to do is now break it into pieces like, storing the data from SD to buffer, checking SD card size. , Test individual code for these and test the functions, and then combine them all.

Do let me know, how it works out. If you face any issue feel free to ask, and meanwhile I’ll try to make up a code for you. (I don’t believe in spoon feeding so I’m asking you to try it once )

Hope this helps!

Cheers!
Parth Temkar

Miha_Horvat · December 6, 2018, 6:25am

Hello @ParthTemkar;
actually it’s not just a diploma project. I work at Water DIstribution System (drinking water) as a student. We take care for whole water supply network. My project (data logger) will measure pressure in pipes. At the same time this project is project for my diploma. So with one project I cover two things - get diploma and create some device/solution for my job.

Thanks for suggestion. Problem here is because I store measurements every second on SD card in log files. We need these log files for backup purposes and also for later data processing (Excel). On server side we visualize data in trends and also store them in archives.

So my SD card’s data will always grow and size will increase.

What is wrong with my solution by storing values to array and in CIPSEND reading from arrays and sending values? It works OK and somehow it solves my primary problem. Please let me know.

Thanks a lot!

RaviPujar · December 6, 2018, 12:50pm

Hi @ParthTemkar Thanks for helping out. Much Appreciated
What i meant is while creating a TCP connection you can create a transparent TCP connection by sending AT+CIPMODE=1 command which will just make the UART communication two way and what you send to module over UART reaches the other side to the server after socket is open and vice versa and you dont need to use CIPSEND or receive command every time.
You can have to use special key sequence to exit from this mode to again send normal AT commands.