Compare commits

..

82 commits

Author SHA1 Message Date
QuentinPerret cb0e33c175 sql integration for create and insert captor metadata and trials ok 2023-06-02 16:35:22 +02:00
QuentinPerret ab1a87c6ac add sql3 in main 2023-06-02 16:10:31 +02:00
QuentinPerret 2443e553c8 add database script in c for creation and init 2023-06-02 15:53:17 +02:00
QuentinPerret 2bd028910f update trail ready 2023-06-02 13:17:44 +02:00
QuentinPerret 29ce8ac9d9 ignore .vscode 2023-06-02 13:03:20 +02:00
QuentinPerret 713b8aae5e remove duration attribut 2023-06-02 13:02:38 +02:00
QuentinPerret 49060a8419 create and insert ok 2023-06-02 12:48:29 +02:00
Quentin Perret 4672159f50 create script ok 2023-06-02 00:03:16 +02:00
QuentinPerret b1a5b8b7bf create table progression 2023-06-01 23:26:13 +02:00
QuentinPerret b15892d0a0 correct includes 2023-06-01 17:22:38 +02:00
QuentinPerret a685866fbc Merge branch 'master' of https://git.cohabit.fr/pgp/Traitement-signal-plantes 2023-05-30 15:48:08 +02:00
quentin.perret 9424c42fca start coding db 2023-05-25 20:20:49 +02:00
Quentin Perret ec3c985534 Folder name Change 2023-05-29 14:42:46 +02:00
QuentinPerret a5ff775863 remove old makefile 2023-05-09 16:31:01 +02:00
QuentinPerret 14107f0301 correct conflit left for Data branch merge , double function definition 2023-05-09 13:46:39 +02:00
QuentinPerret 118504546e merge Data 2023-05-09 13:41:59 +02:00
QuentinPerret 4be1462614 Merge branch 'Data' 2023-05-09 13:41:44 +02:00
QuentinPerret 5be8eaa7c9 change cmake params 2023-05-09 13:37:36 +02:00
quentin.perret 8798202171 change tsl code to easily get values 2023-05-03 10:14:31 +02:00
quentin.perret a037d80c5d add tsl2561 scipt 2023-05-03 09:14:11 +02:00
quentin.perret bf4f35ca29 move .h in include 2022-12-01 17:29:47 +01:00
quentin.perret 011eecc81b beginning code to write json file 2022-09-06 17:33:54 +02:00
quentin.perret 83a308bb96 update gitignore + solve printArrayData issue 2022-09-06 16:44:13 +02:00
Aurélien Gauthier c91ba029e2 add right path image readme 2022-06-28 12:20:11 +02:00
Aurélien Gauthier f86e4380c1 extension type issue 2022-06-28 12:19:03 +02:00
Aurélien Gauthier a95809bfd9 finalisation of readme.md and add image folder 2022-06-28 12:12:50 +02:00
theo.canaud b51a98ba92 graph finished and captor choice possible 2022-06-27 17:31:07 +02:00
Aurélien Gauthier d5cc41f26a test image readme 2022-06-27 16:44:56 +02:00
Aurélien Gauthier 6b9de5f457 rebase merge 2022-06-27 16:39:21 +02:00
Aurélien Gauthier 70e01e558f create readme.md and fix issue on recovery data array from file and adaptative array size frome the number of captors 2022-06-27 16:35:30 +02:00
quentin.perret 1c7d7edaae ad comment for next time fix 2022-06-24 18:30:58 +02:00
Aurélien Gauthier a8c235bc65 first try after first fix strsep 2022-06-24 17:50:54 +02:00
Aurélien Gauthier fecd2124d7 fix strsep 2022-06-24 17:22:33 +02:00
quentin.perret b464ff1541 try to fix getArray 2022-06-24 17:17:28 +02:00
quentin.perret 1eb7ec26b3 create code py for testing calcul 2022-06-24 15:45:25 +02:00
Aurélien Gauthier ee3d1315c0 modif on frequency in simulflux and fix average 2022-06-24 15:42:25 +02:00
Aurélien Gauthier 5e57356f85 add test and modification on queue 2022-06-23 16:24:47 +02:00
theo.canaud 5f23c7b15e added curves for the 8 captors 2022-06-22 16:59:17 +02:00
Aurélien Gauthier 8bd977ba62 fix some warnings 2022-06-21 17:47:27 +02:00
Aurélien Gauthier ade5b4378f CMakelists add few test queue 2022-06-21 15:13:10 +02:00
theo.canaud 8288df18a0 program doesn't stop when error msg appears, instead asks the question again after the error 2022-06-20 17:53:02 +02:00
Aurélien Gauthier f4fd64e63c forget to save main.c 2022-06-17 17:46:53 +02:00
Aurélien Gauthier e1c631229d merge 2022-06-17 17:45:03 +02:00
Aurélien Gauthier 2d58ef1ec3 cmake and begin test 2022-06-17 17:42:09 +02:00
quentin.perret e06c7602e0 add timeFile + ignore last values 2022-06-17 17:38:53 +02:00
theo.canaud 930d91cfa9 plot still appearing even if number not valid fixed 2022-06-17 17:25:14 +02:00
theo.canaud dc3672c56b Added option to choose rawdata used 2022-06-17 16:21:08 +02:00
theo.canaud d63c9cb8df functions added 2022-06-17 10:26:01 +02:00
theo.canaud aafd732599 Tab numpy finished 2022-06-16 17:08:38 +02:00
quentin.perret 8ecdd37619 growth ok 2022-06-16 17:05:12 +02:00
theo.canaud db8baf2d97 Python tab finished 2022-06-16 15:39:25 +02:00
quentin.perret efc3110b2d power debug (column = 0) 2022-06-16 15:19:07 +02:00
quentin.perret 479289a365 vide 2022-06-16 14:44:31 +02:00
quentin.perret e23d744cdc beginning change power 2022-06-16 14:35:02 +02:00
quentin.perret 7af7366247 fix threadboth 2022-06-15 17:50:47 +02:00
quentin.perret c39dcdee1f Merge branch 'master' of https://git.cohabit.fr/pgp/Traitement-signal-plantes 2022-06-15 16:57:53 +02:00
quentin.perret 9030c3df0e go back to : threads clean , captors choice ok 2022-06-15 16:56:58 +02:00
quentin.perret d6f1fc5cfa threads clean , captors choice ok 2022-06-15 16:29:55 +02:00
Aurélien Gauthier 9cf444c47e osef test error 2022-06-15 16:27:56 +02:00
theo.canaud 7cc7533214 RawDataFiles changed, start of tab 2022-06-15 16:25:43 +02:00
quentin.perret ab66c1649b def function printPlot 2022-06-15 15:46:40 +02:00
Aurélien Gauthier 21cb319b25 growthRate thread implémentation 2022-06-15 15:45:14 +02:00
Aurélien Gauthier 3435430c02 Merge branch 'Data' of ssh://git.cohabit.fr:22222/pgp/Traitement-signal-plantes into Data 2022-06-15 14:02:10 +02:00
Aurélien Gauthier 0c63639454 add in main.c 2022-06-15 14:01:55 +02:00
quentin.perret a6c0e54756 add comment for work in main 2022-06-15 13:59:27 +02:00
Aurélien Gauthier c66535111d merge add thread growth rate 2022-06-15 12:03:46 +02:00
Aurélien Gauthier 7d024ff384 first part and skeletton of growthrate thread 2022-06-15 12:01:51 +02:00
quentin.perret 9cd8d2bc7e add new thread growth rate 2022-06-15 11:57:15 +02:00
quentin.perret 33115049f6 threads clean , captors choice ok 2022-06-15 10:49:21 +02:00
quentin.perret f9a6f00972 trying to debug power 2022-06-14 18:01:52 +02:00
theo.canaud a35e1a8adf Merge branch 'master' of https://git.cohabit.fr/pgp/Traitement-signal-plantes 2022-06-14 14:44:36 +02:00
theo.canaud e01d5b553b Limites du graphique retirer 2022-06-14 14:43:43 +02:00
quentin.perret 9bb5e5e99d Merge branch 'SimulFlux' 2022-06-14 11:42:00 +02:00
theo.canaud 16948e901b Courbe seul 2022-06-14 11:16:54 +02:00
theo.canaud 13b07003ee Merge branch 'master' of https://git.cohabit.fr/pgp/Traitement-signal-plantes 2022-06-13 15:24:05 +02:00
theo.canaud 8906063068 Tableau une seul courbe 2022-06-13 15:22:50 +02:00
theo.canaud c0e70cec4e Merge branch 'master' of https://git.cohabit.fr/pgp/Traitement-signal-plantes 2022-06-13 15:22:05 +02:00
quentin.perret af985c81df Merge branch 'SimulFlux' 2022-06-13 15:20:21 +02:00
quentin.perret 2e1462a0ff adapt code for pack 2022-06-13 15:12:07 +02:00
quentin.perret ceacb1c512 Merge branch 'SimulFlux' 2022-06-13 15:10:22 +02:00
theo.canaud cde4670a1c Tableau de données avec une seul courbe 2022-06-13 15:07:02 +02:00
theo.canaud 926016943f Ajout de la branch courbe 2022-06-07 16:22:49 +02:00
47 changed files with 2883 additions and 451 deletions

92
.gitignore vendored
View file

@ -2,3 +2,95 @@
*.png *.png
*.exe *.exe
*.csv *.csv
*.png
*.dia
Code-C/main
Code-C/exect
Code-C/ctest
Code-C/DartConfiguration.tcl
Code-C/tsl
Makefile
.vscode
# Created by https://www.toptal.com/developers/gitignore/api/cmake,c
# Edit at https://www.toptal.com/developers/gitignore?templates=cmake,c
### C ###
# Prerequisites
*.d
# Object files
*.o
*.ko
*.obj
*.elf
# Linker output
*.ilk
*.map
*.exp
# Precompiled Headers
*.gch
*.pch
# Libraries
*.lib
*.a
*.la
*.lo
# Shared objects (inc. Windows DLLs)
*.dll
*.so
*.so.*
*.dylib
# Executables
*.exe
*.out
*.app
*.i*86
*.x86_64
*.hex
*/Executable
# Debug files
*.dSYM/
*.su
*.idb
*.pdb
# Kernel Module Compile Results
*.mod*
*.cmd
.tmp_versions/
modules.order
Module.symvers
Mkfile.old
dkms.conf
### CMake ###
CMakeLists.txt.user
CMakeCache.txt
CMakeFiles
CMakeScripts
Testing
Makefile
cmake_install.cmake
install_manifest.txt
compile_commands.json
CTestTestfile.cmake
_deps
build
### CMake Patch ###
# External projects
*-prefix/
### DataFiles ###
*/RawDataFiles
# End of https://www.toptal.com/developers/gitignore/api/cmake,c

20
.vscode/settings.json vendored
View file

@ -1,20 +0,0 @@
{
"C_Cpp.errorSquiggles": "Disabled",
"files.associations": {
"time.h": "c",
"b2hd.h": "c",
"main.h": "c",
"stdio.h": "c",
"string.h": "c",
"initialparameters.h": "c",
"filegestion.h": "c",
"power.h": "c",
"getarray.h": "c",
"math.h": "c",
"limits": "c",
"*.tcc": "c",
"type_traits": "c",
"simulateflux.h": "c",
"pthread.h": "c"
}
}

20
.vscode/tasks.json vendored
View file

@ -1,20 +0,0 @@
{
"tasks": [
{
"type": "cppbuild",
"label": "C/C++: gcc.exe build active file",
"command": "C:/msys64/mingw64/bin/gcc.exe",
"args": ["-g", "${file}", "-o", "${fileDirname}\\${fileBasenameNoExtension}"],
"options": {
"cwd": "${fileDirname}"
},
"problemMatcher": ["$gcc"],
"group": {
"kind": "build",
"isDefault": true
},
"detail": "compiler: C:/msys64/mingw64/bin/gcc.exe"
}
],
"version": "2.0.0"
}

View file

@ -1,5 +0,0 @@
{
"files.associations": {
"power.h": "c"
}
}

47
Code-C/CMakeLists.txt Normal file
View file

@ -0,0 +1,47 @@
cmake_minimum_required(VERSION 2.8)
project(Traitement-signal-plantes C)
include(CTest)
enable_testing()
set(CMAKE_C_FLAGS "-std=c99 -g -Wall") #what are the flag for ?
file(MAKE_DIRECTORY RawDataFiles) #why do we make this folder each time ?
file(MAKE_DIRECTORY Executable)
set (CMAKE_RUNTIME_OUTPUT_DIRECTORY Executable) #Set Executable directory as default exectutable file location
add_executable(exect fileGestion.c getArray.c average.c growthRate.c power.c queue.c simulateFlux.c database.c main.c)
# add_executable(exect main.c simulateFlux.c queue.c power.c growthRate.c average.c getArray.c fileGestion.c)
find_package(SQLite3 REQUIRED)
target_link_libraries(exect ${SQLite3_LIBRARIES} m)
find_package(Threads)
target_link_libraries(exect ${CMAKE_THREAD_LIBS_INIT} m)
set(CTEST_MEMORYCHECK_TYPE "AddressSanitizer")
set(CTEST_MEMORYCHECK_SANITIZER_OPTIONS "verbosity=1:symbolize=1:abort_on_error=1:detect_leaks=1")
add_executable(ctest fileGestion.c getArray.c average.c growthRate.c queue.c simulateFlux.c database.c ctest.c)
target_link_libraries(ctest ${SQLite3_LIBRARIES} m)
target_link_libraries(ctest ${CMAKE_THREAD_LIBS_INIT} m)
add_test(test_queueCreateEmpty ./ctest queueCreateEmpty)
add_test(test_queueCreateE ./ctest queueCreateE)
add_test(test_queueSetCharLen ./ctest queueSetCharLen)
add_test(test_queueGetCharLen ./ctest queueGetCharLen)
add_test(test_queueSetTabChar ./ctest queueSetTabChar)
add_test(test_queueGetTabChar ./ctest queueGetTabChar)
add_test(test_queueSetNextE ./ctest queueSetNextE)
add_test(test_queueGetNextE ./ctest queueGetNextE)
add_test(test_queueAddLastQ ./ctest queueAddLastQ)
add_test(test_queueRmLastE ./ctest queueRmLastE)
add_test(test_queueRmFrstE ./ctest queueRmFrstE)
add_test(test_queueNextDelFrst ./ctest queueNextDelFrst)

14
Code-C/Include/TSL2561.h Normal file
View file

@ -0,0 +1,14 @@
#include <stdio.h>
#include <stdlib.h>
#include <linux/i2c-dev.h>
#include <sys/ioctl.h>
#include <fcntl.h>
#include <unistd.h>
char *bus = "/dev/i2c-1";
void printSpectrum(int file);
float getFullSpectrum(int file);
float getInfraredLight(int file);
int init_TSL2561(char *bus);

4
Code-C/Include/average.h Normal file
View file

@ -0,0 +1,4 @@
#include <math.h>
#include <stdbool.h>
void averageFunction(char* rawDataFileName , double **aver);

View file

@ -2,23 +2,19 @@
#include <string.h> #include <string.h>
#include <stdint.h> #include <stdint.h>
#include <stdlib.h> #include <stdlib.h>
#include <time.h> //#include <linuxstruct_timespec.h>
#define __USE_ISOC11 1
#include <linux/time.h>
#include <math.h> #include <math.h>
/** /**
* @brief struct used to stock binary 32 bits data from the captor * @brief struct used to stock binary 32 bits data from the captor
* *
*/ */
typedef struct { typedef struct
{
uint8_t octet1; uint8_t octet1;
uint8_t octet2; uint8_t octet2;
uint8_t octet3; uint8_t octet3;
uint8_t octet4; uint8_t octet4;
} quartet; } quartet;
int64_t millis()
{
struct timespec now;
timespec_get(&now, TIME_UTC);
return ((int64_t) now.tv_sec) * 1000 + ((int64_t) now.tv_nsec) / 1000000;
}

11
Code-C/Include/database.h Normal file
View file

@ -0,0 +1,11 @@
#include <sqlite3.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <stdbool.h>
#include <unistd.h>
#include <math.h>
int createDb();
int initiaizeNewTrial(char *position, double frequency, bool *captorOneHot);
int initiaizeNewTrial(char *position, double frequency, bool *captorOneHot);

View file

@ -5,4 +5,4 @@
void clearRawData(int N); void clearRawData(int N);
void writeDataInFile(char* fileName , double a[], int N); void appendDataInFile(char* fileName , double a[], int N);

12
Code-C/Include/getArray.h Normal file
View file

@ -0,0 +1,12 @@
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <stdlib.h>
#include <stdbool.h>
#include <inttypes.h>
long **getRawDataArray(char *rawDataFileName);
void printArrayData(long **p , int N , int M);
void freeArray(long **p, int N);
bool checkArrayFullyFill(long **p, int N);
double **getDoubleArray(int N, int M);

View file

@ -0,0 +1,4 @@
#include <math.h>
#include <stdbool.h>
void growthRateFunction(double **dataLign , char* fileName);

View file

@ -2,14 +2,18 @@
#include "queue.h" #include "queue.h"
extern bool rawDataWriteFlag; extern bool rawDataWriteFlag;
extern int nRow; extern int nRowRawData;
extern int nRowGR;
extern int nCol; extern int nCol;
extern double freqEch; extern double freqEch;
extern int nbRowBinFile;
extern int nbRowIgnore;
extern int cptFile;
extern int cptValue;
extern double period; extern double period;
extern double invTimeBandWidth; extern double invTimeBandWidth;
extern int selectionCaptors[]; extern bool selectionCaptors[];
extern int sizeSelectionArray;
extern Pqueue firstRawDataQueue; extern Pqueue firstRawDataQueue;

4
Code-C/Include/power.h Normal file
View file

@ -0,0 +1,4 @@
#include <math.h>
#include <stdbool.h>
void powerFunction(char* rawDataFileName, double *pw[]);

127
Code-C/Include/queue.h Normal file
View file

@ -0,0 +1,127 @@
typedef struct queue *Pqueue;
/**
* @brief Create Element of queue
*
* @param lenChar size of char array that will be created
* @return Pqueue new element created
*/
Pqueue queueCreateE(int lenChar, const char *_tabChar);
/**
* @brief create an empty element of queue
*
* @return Pqueue
*/
Pqueue queueCreateEmpty();
/************** Getters **************/
/**
* @brief switch to the next element
*
* @param elem current element
* @return Pqueue next element
*/
Pqueue queueGetNextE(Pqueue elem);
/**
* @brief get the size char array
*
* @param elem targeted element
* @return int
*/
int queueGetCharLen(Pqueue elem);
/**
* @brief get the pointer of char array
*
* @param elem targeted elemnt
* @return char*
*/
char *queueGetTabChar(Pqueue elem);
/************** Setters **************/
/**
* @brief set char size array
*
* @param elem targeted element
* @param _charLen size of char array
*/
void queueSetCharLen(Pqueue elem, int _charLen);
/**
* @brief set the char array in the element
*
* @param elem targeted element
* @param _charLen char array size
* @param _tabChar pointer to static char array
*/
void queueSetTabChar(Pqueue elem, int _charLen, const char *_tabChar);
/**
* @brief set next pqueue element
*
* @param elem target element
* @param next next element
*/
void queueSetNextE(Pqueue elem, Pqueue next);
/**
* @brief add in the end of queue element with string parameter
*
* @param elem Pqueue will be added at the end of queue
* @param str the string will be bind at the element
* @param len the lenght of char array
*/
void queueAddLastQ(Pqueue elem, const char *str, int len);
/************** Removers **************/
/**
* @brief remove and free the last element of queue
*
* @param elem current element
* @return Pqueue current element
*/
Pqueue queueRmLastE(Pqueue elem);
/**
* @brief remove and free the first element of queue
*
* @param elem target to remove (the element need to be the first)
* @return the next element of elem
*/
Pqueue queueRmFrstE(Pqueue elem); // same of queueNextDelFrst()
/**
* @brief delete the first value and return the next value
*
* @param elem
* @return Pqueue
*/
Pqueue queueNextDelFrst(Pqueue elem); // same of queueRmFrstE()
/************** print function **************/
/**
* @brief Print targeted element
*
* @param elem targeted element
*/
void queuePrintE(Pqueue elem);
/**
* @brief Print all the element starting from @param firstE till the end of the linked list
*
* @param firstE
*/
void queuePrintWholeQ(Pqueue firstE);
/**
* @brief free all queue list
*
* @param elem the first element of comdemned queue
*/
void queueDelAll(Pqueue elem);

View file

@ -4,6 +4,7 @@
#include <stdlib.h> #include <stdlib.h>
#include <unistd.h> #include <unistd.h>
#include <stdbool.h> #include <stdbool.h>
#define __USE_ISOC11 1
#include <time.h> #include <time.h>
typedef struct { typedef struct {

View file

@ -1,7 +0,0 @@
CC = gcc
all:
$(CC) fileGestion.c getArray.c average.c power.c queue.c simulateFlux.c main.c -lm -lpthread -o main
# $(CC) queue.c simulateFlux.c main.c -lm -lpthread -o main
./main < ../02400031.TXT

108
Code-C/TSL2561.c Normal file
View file

@ -0,0 +1,108 @@
// Code from the following github repo: https://github.com/ControlEverythingCommunity/TSL2561/blob/master/C/TSL2561.c
#include "./Include/TSL2561.h"
void printSpectrum(int file)
{
// Read 4 bytes of data from register(0x0C | 0x80)
// ch0 lsb, ch0 msb, ch1 lsb, ch1 msb
char reg[1] = {0x0C | 0x80};
write(file, reg, 1);
char data[4] = {0};
if (read(file, data, 4) != 4)
{
printf("Erorr : Input/output Erorr \n");
}
else
{
// Convert the data
float ch0 = (data[1] * 256 + data[0]); // Full Spectrum(IR + Visible)
float ch1 = (data[3] * 256 + data[2]); // Infrared Value
// Output data to screen
printf("Full Spectrum(IR + Visible) : %.1f lux \n", ch0);
printf("Infrared Value : %.1f lux \n", ch1);
printf("Visible Value : %.1f lux \n", (ch0 - ch1));
}
}
float getFullSpectrum(int file)
{
// Read 4 bytes of data from register(0x0C | 0x80)
// ch0 lsb, ch0 msb, ch1 lsb, ch1 msb
char reg[1] = {0x0C | 0x80};
write(file, reg, 1);
char data[4] = {0};
if (read(file, data, 4) != 4)
{
printf("Erorr : Input/output Erorr \n");
}
else
{
// Convert the data
float ch0 = (data[1] * 256 + data[0]); // Full Spectrum(IR + Visible)
return ch0;
}
}
float getInfraredLight(int file)
{
// Read 4 bytes of data from register(0x0C | 0x80)
// ch0 lsb, ch0 msb, ch1 lsb, ch1 msb
char reg[1] = {0x0C | 0x80};
write(file, reg, 1);
char data[4] = {0};
if (read(file, data, 4) != 4)
{
printf("Erorr : Input/output Erorr \n");
}
else
{
// Convert the data
float ch1 = (data[3] * 256 + data[2]); // Infrared Value
return ch1;
}
}
int init_TSL2561(char *bus)
{
int file;
if ((file = open(bus, O_RDWR)) < 0)
{
printf("Failed to open the bus. \n");
exit(1);
}
// Get I2C device, TSL2561 I2C address is 0x39(57)
ioctl(file, I2C_SLAVE, 0x39);
// Select control register(0x00 | 0x80)
// Power ON mode(0x03)
char config[2] = {0};
config[0] = 0x00 | 0x80;
config[1] = 0x03;
write(file, config, 2);
// Select timing register(0x01 | 0x80)
// Nominal integration time = 402ms(0x02)
config[0] = 0x01 | 0x80;
config[1] = 0x02;
write(file, config, 2);
return file;
}
void TSL2561()
{
int fe = 1;
int file;
file = init_TSL2561(bus);
while (1)
{
sleep(1 / fe);
float a = getFullSpectrum(file);
float b = getInfraredLight(file);
printf("%.1f\n", a);
printf("%.1f\n", b);
printSpectrum(file);
}
}
void main()
{
TSL2561();
}

View file

@ -1,26 +1,28 @@
#include "average.h" #include "./Include/average.h"
#include "getArray.h" #include "./Include/getArray.h"
#include "fileGestion.h" #include "./Include/fileGestion.h"
#include "initialParameters.h" #include "./Include/initialParameters.h"
#include "queue.h" #include "./Include/queue.h"
/** /**
* @brief realize the average calcul * @brief realize the average calcul
* *
* @param p array with all the values that will be used for the calcul * @param p array with all the values that will be used for the calcul
* @param averageArray array where results are stocked * @param averageArray array where results are stocked
* @param N number of rows in p
* @param M number of columns in p
*/ */
void averageCalculation(long **p, double averageArray[] , int N, int M){ void averageCalculation(long **p, double averageArray[])
for(int i = 0; i < M-1; i++){ {
// printArrayData(p, nRowRawData, nCol);
// printf("\n");
for (int i = 1; i < nCol; i++)
{
int j = 0; int j = 0;
averageArray[i] = 0; averageArray[i] = 0;
while(j < N){ while (j < nRowRawData)
averageArray[i] += p[i][j]; {
averageArray[i - 1] += (double)p[i][j];
j++; j++;
} }
averageArray[i] /= N; averageArray[i] /= nRowRawData;
//printf("%f\n", powerArray[i]);
} }
} }
@ -28,15 +30,26 @@ void averageCalculation(long **p, double averageArray[] , int N, int M){
* @brief function that realize all the action to write one lign in the file averageData.csv * @brief function that realize all the action to write one lign in the file averageData.csv
* *
* @param rawDataFileName name of the raw data file to use to realize the calcul * @param rawDataFileName name of the raw data file to use to realize the calcul
* @param N number of rows in the file
* @param M number of columns in the file
*/ */
void average(char* rawDataFileName,int N , int M){ void averageFunction(char *rawDataFileName, double **aver)
long **p = getRawDataArray(rawDataFileName,N, M); {
double aver[8]; long **p = getRawDataArray(rawDataFileName);
if(p !=NULL){ double averN[nCol - 1];
averageCalculation(p,aver,N,M); if (p != NULL)
writeDataInFile("averageData.csv",aver,8); {
freeArray(p,N); if (p != NULL)
{
if (aver == NULL)
{
averageCalculation(p, averN);
appendDataInFile("averageData.csv", averN, nCol - 1);
}
else
{
averageCalculation(p, aver[1]);
appendDataInFile("averageData.csv", aver[1], nCol - 1);
}
freeArray(p, nRowRawData);
}
} }
} }

View file

@ -1,4 +0,0 @@
#include <math.h>
#include <stdbool.h>
void average(char* rawDataFileName,int N , int M);

View file

@ -1,4 +1,19 @@
#include "b2hd.h" #include "./Include/b2hd.h"
#include <time.h>
int64_t millis()
{
// struct timespec now;
struct timespec *now = (struct timespec *)malloc(sizeof(struct timespec));
// timespec_get(&now, TIME_UTC);
timespec_get(now, TIME_UTC);
// #ifdef __USE_ISOC11 ????
int64_t tmp = (((int64_t)now->tv_sec) * 1000 + ((int64_t)now->tv_nsec) / ((int64_t)1000000));
free(now);
return tmp;
}
/** /**
* @brief allow to transform all binary hex data send by the Vegetal Signal captor in a .TXT file into decimal values in a .csv * @brief allow to transform all binary hex data send by the Vegetal Signal captor in a .TXT file into decimal values in a .csv
* *
@ -13,38 +28,46 @@ void b2hd()
uint32_t valbin[8]; uint32_t valbin[8];
quartet value; quartet value;
while (fread(&buff, 26, 1, stdin)) { while (fread(&buff, 26, 1, stdin))
fprintf(stdout , "%d,", millis()); {
for (int i = 1; i < 9; i++){ fprintf(stdout, "%ld,", millis());
for (int i = 1; i < 9; i++)
{
/*buff[25] = '\0';*/ /*buff[25] = '\0';*/
if (strncmp(buff, "#################\n", (size_t)18) == 0) { if (strncmp(buff, "#################\n", (size_t)18) == 0)
if (!(fread(&buff2, 18, 1, stdin))) { {
if (!(fread(&buff2, 18, 1, stdin)))
{
fprintf(stderr, "Erreur lesture après ###...#"); fprintf(stderr, "Erreur lesture après ###...#");
break; break;
} else { }
else
{
/*buff2[17] = '\0';*/ /*buff2[17] = '\0';*/
strncpy(buff, &buff[18], 8); strncpy(buff, &buff[18], 8);
strncpy(&buff[8], buff2, 18); strncpy(&buff[8], buff2, 18);
} }
} }
value.octet1 = buff[3*i+1]; value.octet1 = buff[3 * i + 1];
value.octet2 = buff[3*i+2]; value.octet2 = buff[3 * i + 2];
value.octet3 = buff[3*i+3]; value.octet3 = buff[3 * i + 3];
value.octet4 = 0; value.octet4 = 0;
/*memcpy(&values[i], &value, sizeof(quartet));*/ /*memcpy(&values[i], &value, sizeof(quartet));*/
valbin[i] = buff[3*i+1]*256*256*256 + buff[3*i+2]*256*256 + buff[3*i+3]*256; valbin[i] = buff[3 * i + 1] * 256 * 256 * 256 + buff[3 * i + 2] * 256 * 256 + buff[3 * i + 3] * 256;
memcpy(&values[i], &valbin[i], sizeof(uint32_t)); memcpy(&values[i], &valbin[i], sizeof(uint32_t));
if(i<8){ if (i < 8)
fprintf(stdout, "%d,", values[i]/256); {
fprintf(stdout, "%d,", values[i] / 256);
} }
else{ else
fprintf(stdout, "%d\n", values[i]/256); {
fprintf(stdout, "%d\n", values[i] / 256);
} }
} }
} }
} }
int main(int argc, char **argv)
int main(int argc , char** argv){ {
b2hd(); b2hd();
} }

View file

@ -0,0 +1,37 @@
# Copyright (C) 2007-2009 LuaDist.
# Created by Peter Kapec <kapecp@gmail.com>
# Redistribution and use of this file is allowed according to the terms of the MIT license.
# For details see the COPYRIGHT file distributed with LuaDist.
# Note:
# Searching headers and libraries is very simple and is NOT as powerful as scripts
# distributed with CMake, because LuaDist defines directories to search for.
# Everyone is encouraged to contact the author with improvements. Maybe this file
# becomes part of CMake distribution sometimes.
# - Find sqlite3
# Find the native SQLITE3 headers and libraries.
#
# SQLITE3_INCLUDE_DIRS - where to find sqlite3.h, etc.
# SQLITE3_LIBRARIES - List of libraries when using sqlite.
# SQLITE3_FOUND - True if sqlite found.
# Look for the header file.
FIND_PATH(SQLITE3_INCLUDE_DIR NAMES sqlite3.h)
# Look for the library.
FIND_LIBRARY(SQLITE3_LIBRARY NAMES sqlite)
# Handle the QUIETLY and REQUIRED arguments and set SQLITE3_FOUND to TRUE if all listed variables are TRUE.
INCLUDE(FindPackageHandleStandardArgs)
FIND_PACKAGE_HANDLE_STANDARD_ARGS(SQLITE3 DEFAULT_MSG SQLITE3_LIBRARY SQLITE3_INCLUDE_DIR)
# Copy the results to the output variables.
IF(SQLITE3_FOUND)
SET(SQLITE3_LIBRARIES ${SQLITE3_LIBRARY})
SET(SQLITE3_INCLUDE_DIRS ${SQLITE3_INCLUDE_DIR})
ELSE(SQLITE3_FOUND)
SET(SQLITE3_LIBRARIES)
SET(SQLITE3_INCLUDE_DIRS)
ENDIF(SQLITE3_FOUND)
MARK_AS_ADVANCED(SQLITE3_INCLUDE_DIRS SQLITE3_LIBRARIES)

308
Code-C/ctest.c Normal file
View file

@ -0,0 +1,308 @@
#define __USE_GNU
#include <assert.h>
#include <stdbool.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "./Include/fileGestion.h"
#include "./Include/getArray.h"
#include "./Include/growthRate.h"
#include "./Include/power.h"
#include "./Include/simulateFlux.h"
#include "./Include/queue.h"
#include "./Include/average.h"
// #include "b2hd.h"
bool rawDataWriteFlag;
int nRowRawData = 500;
int nRowGR = 150;
int nCol = 1;
double freqEch = 250;
int nbRowBinFile = 900011;
int nbRowIgnore = 19;
Pqueue firstRawDataQueue;
// Captor 1 2 3 4 5 6 7 8
bool selectionCaptors[] = {true, false, true, false, false, false, true, false};
int cptData = 0;
int cptFile = 1;
int cptValue = 0;
double period = 0;
double invTimeBandWidth = 0;
bool test_queueCreateEmpty()
{
Pqueue new = queueCreateEmpty();
assert(queueGetCharLen(new) == 0);
assert(queueGetTabChar(new) == NULL);
assert(queueGetNextE(new) == NULL);
// queueDelAll(new);
queueRmFrstE(new);
return EXIT_SUCCESS;
}
bool test_queueCreateE()
{
const char tabChar[12] = "PetitNavire\0";
Pqueue new = queueCreateE(12, tabChar);
const char *test = queueGetTabChar(new);
assert(queueGetCharLen(new) == 12);
assert(test != NULL);
assert(strcmp(test, tabChar) == 0);
assert(queueGetNextE(new) == NULL);
queueRmFrstE(new);
return EXIT_SUCCESS;
}
bool test_queueSetCharLen()
{
Pqueue new = queueCreateEmpty();
queueSetCharLen(new, 13);
assert(queueGetCharLen(new) == 13);
assert(queueGetTabChar(new) == NULL);
assert(queueGetNextE(new) == NULL);
// queueDelAll(new);
queueRmFrstE(new);
return EXIT_SUCCESS;
}
bool test_queueSetTabChar()
{
// Set and Get tabChar are similar because use for same test
Pqueue new = queueCreateEmpty();
const char tabChar[12] = "PetitNavire\0";
queueSetTabChar(new, 12, tabChar);
const char *test = queueGetTabChar(new);
assert(queueGetCharLen(new) == 12);
assert(test != NULL);
assert(strcmp(test, tabChar) == 0);
assert(queueGetNextE(new) == NULL);
queueRmFrstE(new);
return EXIT_SUCCESS;
}
bool test_queueGetTabChar()
{
Pqueue new = queueCreateEmpty();
const char tabChar[12] = "PetitNavire\0";
queueSetTabChar(new, 12, tabChar);
const char *test = queueGetTabChar(new);
assert(queueGetCharLen(new) == 12);
assert(test != NULL);
assert(strcmp(test, tabChar) == 0);
assert(queueGetNextE(new) == NULL);
queueRmFrstE(new);
return EXIT_SUCCESS;
}
bool test_queueSetNextE()
{
Pqueue new = queueCreateEmpty(), next = queueCreateEmpty();
queueSetNextE(new, next);
assert(queueGetNextE(new) == next);
assert(queueGetNextE(next) == NULL);
queueDelAll(new);
return EXIT_SUCCESS;
}
bool test_queueGetNextE()
{
Pqueue new = queueCreateEmpty(), next = queueCreateEmpty();
queueSetNextE(new, next);
assert(queueGetNextE(new) == next);
assert(queueGetNextE(next) == NULL);
queueDelAll(new);
return EXIT_SUCCESS;
}
bool test_queueGetCharLen()
{
Pqueue new = queueCreateEmpty();
assert(queueGetCharLen(new) == 0);
queueSetCharLen(new, 13);
assert(queueGetCharLen(new) == 13);
queueDelAll(new);
return EXIT_SUCCESS;
}
bool test_queueAddLastQ()
{
Pqueue new = queueCreateEmpty(), next = NULL;
const char tabChar[12] = "PetitNavire\0";
queueAddLastQ(new, tabChar, 12);
next = queueGetNextE(new);
assert(next != NULL);
assert(queueGetCharLen(next) == 12);
assert(strcmp(queueGetTabChar(next), tabChar) == 0);
assert(queueGetNextE(next) == NULL);
queueDelAll(new);
return EXIT_SUCCESS;
}
bool test_queueRmLastE()
{
Pqueue new = queueCreateEmpty();
const char tabChar[12] = "PetitNavire\0";
queueAddLastQ(new, tabChar, 12);
queueAddLastQ(new, tabChar, 12);
assert(queueGetNextE(new) != NULL && queueGetNextE(queueGetNextE(new)) != NULL && queueGetNextE(queueGetNextE(queueGetNextE(new))) == NULL);
queueRmLastE(new);
assert(queueGetNextE(new) != NULL && queueGetNextE(queueGetNextE(new)) == NULL);
queueDelAll(new);
return EXIT_SUCCESS;
}
bool test_queueRmFrstE()
{
Pqueue new = queueCreateEmpty(), second;
const char tabChar[12] = "PetitNavire\0";
queueAddLastQ(new, tabChar, 12);
queueAddLastQ(new, tabChar, 12);
second = queueGetNextE(new);
assert(queueGetNextE(new) != NULL && queueGetNextE(queueGetNextE(new)) != NULL && queueGetNextE(queueGetNextE(queueGetNextE(new))) == NULL);
new = queueRmFrstE(new);
assert((new == second) && (queueGetNextE(new) != NULL) && (queueGetNextE(queueGetNextE(new)) == NULL));
queueDelAll(new);
return EXIT_SUCCESS;
}
bool test_queueNextDelFrst()
{
Pqueue new = queueCreateEmpty(), second;
const char tabChar[12] = "PetitNavire\0";
queueAddLastQ(new, tabChar, 12);
queueAddLastQ(new, tabChar, 12);
second = queueGetNextE(new);
assert(queueGetNextE(new) != NULL && queueGetNextE(queueGetNextE(new)) != NULL && queueGetNextE(queueGetNextE(queueGetNextE(new))) == NULL);
new = queueNextDelFrst(new);
assert((new == second) && (queueGetNextE(new) != NULL) && (queueGetNextE(queueGetNextE(new)) == NULL));
queueDelAll(new);
return EXIT_SUCCESS;
}
/***************** Main test start *****************/
void usage(int argc, char *argv[])
{
fprintf(stderr, "Usage: %s <testname> [<...>]\n", argv[0]);
exit(EXIT_FAILURE);
}
int main(int argc, char *argv[])
{
if (argc == 1)
usage(argc, argv);
// start test
fprintf(stderr, "=> Start test \"%s\"\n", argv[1]);
int ok = 1;
if (strcmp("queueCreateEmpty", argv[1]) == 0)
{
ok = test_queueCreateEmpty();
}
else if (strcmp("queueCreateE", argv[1]) == 0)
{
ok = test_queueCreateE();
}
else if (strcmp("queueSetCharLen", argv[1]) == 0)
{
ok = test_queueSetCharLen();
}
else if (strcmp("queueGetTabChar", argv[1]) == 0)
{
ok = test_queueGetTabChar();
}
else if (strcmp("queueSetNextE", argv[1]) == 0)
{
ok = test_queueSetNextE();
}
else if (strcmp("queueGetNextE", argv[1]) == 0)
{
ok = test_queueGetNextE();
}
else if (strcmp("queueGetCharLen", argv[1]) == 0)
{
ok = test_queueGetCharLen();
}
else if (strcmp("queueGetTabChar", argv[1]) == 0)
{
ok = test_queueGetTabChar();
}
else if (strcmp("queueSetTabChar", argv[1]) == 0)
{
ok = test_queueSetTabChar();
}
else if (strcmp("queueAddLastQ", argv[1]) == 0)
{
ok = test_queueAddLastQ();
}
else if (strcmp("queueRmLastE", argv[1]) == 0)
{
ok = test_queueRmLastE();
}
else if (strcmp("queueRmFrstE", argv[1]) == 0)
{
ok = test_queueRmFrstE();
}
else if (strcmp("queueNextDelFrst", argv[1]) == 0)
{
ok = test_queueNextDelFrst();
// }else if(strcmp("default_solution",argv[1]) == 0){
// ok = test_game_default_solution();
// }else if(strcmp("default_solution",argv[1]) == 0){
// ok = test_game_default_solution();
}
else
{
fprintf(stderr, "Error: test \"%s\" not found!\n", argv[1]);
exit(EXIT_FAILURE);
}
// print test result
if (ok == 0)
{
fprintf(stderr, "Test \"%s\" finished: SUCCESS\n", argv[1]);
return EXIT_SUCCESS;
}
else
{
fprintf(stderr, "Test \"%s\" finished: FAILURE\n", argv[1]);
return EXIT_FAILURE;
}
}

248
Code-C/database.c Normal file
View file

@ -0,0 +1,248 @@
#include "./Include/simulateFlux.h"
#include "./Include/database.h"
int initializeTrialsTable()
{
sqlite3 *db;
int rc;
sqlite3_stmt *stmt;
rc = sqlite3_open("robotgowest.db", &db);
// char *sql = "CREATE TABLE Test(Id INT)";
char *sql =
"CREATE TABLE Trials(Id INTEGER PRIMARY KEY AUTOINCREMENT, Position TEXT, Start INT, Frequency NUMBER);";
rc = sqlite3_prepare_v2(db, sql, -1, &stmt, NULL);
rc = sqlite3_step(stmt);
rc = sqlite3_finalize(stmt);
sqlite3_close(db);
return 0;
}
int initializeCaptorMetadataTable()
{
sqlite3 *db;
int rc;
sqlite3_stmt *stmt;
rc = sqlite3_open("robotgowest.db", &db);
// char *sql = "CREATE TABLE Test(Id INT)";
char *sql =
"CREATE TABLE CaptorMetadata(Id INTEGER PRIMARY KEY AUTOINCREMENT, CaptorNumber INT, TrialId INT, FOREIGN KEY (TrialId) REFERENCES Trials(Id));";
rc = sqlite3_prepare_v2(db, sql, -1, &stmt, NULL);
rc = sqlite3_step(stmt);
rc = sqlite3_finalize(stmt);
sqlite3_close(db);
return 0;
}
char *generateDataTableName(int trialId, int captorNumber)
{
char *tableName;
tableName = (char *)malloc(8 * sizeof(char));
strcpy(tableName, "T");
strcat(tableName, convertIntegerToChar(trialId));
strcat(tableName, "C");
strcat(tableName, convertIntegerToChar(captorNumber));
strcat(tableName, "Data");
return tableName;
}
int createCaptorDataTable(int trialId, int captorNumber)
{
sqlite3 *db;
int rc;
sqlite3_stmt *stmt;
// Create Table Name
char *tableName = generateDataTableName(trialId, captorNumber);
rc = sqlite3_open("robotgowest.db", &db);
if (rc != SQLITE_OK)
{
return 1;
}
char sql[101] = "CREATE TABLE ";
strcat(sql, tableName);
strcat(sql, " (Id INTEGER PRIMARY KEY AUTOINCREMENT, Lux NUMBER, Mean NUMBER , Power NUMBER);");
rc = sqlite3_prepare_v2(db, sql, -1, &stmt, NULL);
rc = sqlite3_step(stmt);
rc = sqlite3_finalize(stmt);
sqlite3_close(db);
return 0;
}
int insertTrailElement(char *position, int start, double frequency)
{
sqlite3 *db;
sqlite3_stmt *stmt;
int rc;
rc = sqlite3_open("robotgowest.db", &db);
rc = sqlite3_prepare_v2(db, "INSERT INTO Trials(Position,Start,Frequency) VALUES ( :position , :start , :frequency )", -1, &stmt, NULL);
if (rc != SQLITE_OK)
return 1;
sqlite3_bind_text(stmt, 1, position, -1, SQLITE_STATIC);
sqlite3_bind_int(stmt, 2, start);
sqlite3_bind_double(stmt, 3, frequency);
rc = sqlite3_step(stmt);
rc = sqlite3_finalize(stmt);
sqlite3_close(db);
return 0;
}
int updateTrailElement(int id, char *position, int start, double frequency)
{
sqlite3 *db;
sqlite3_stmt *stmt;
int rc;
rc = sqlite3_open("robotgowest.db", &db);
rc = sqlite3_prepare_v2(db, "UPDATE Trials SET Position = :position, Start = :start, Frequency = :frequency WHERE Id = :id", -1, &stmt, NULL);
if (rc != SQLITE_OK)
return 1;
sqlite3_bind_text(stmt, 1, position, -1, SQLITE_STATIC);
sqlite3_bind_int(stmt, 2, start);
sqlite3_bind_double(stmt, 3, frequency);
sqlite3_bind_int(stmt, 4, id);
rc = sqlite3_step(stmt);
rc = sqlite3_finalize(stmt);
sqlite3_close(db);
return 0;
}
int insertCaptorMetadataElement(int captorNumber, int trialId)
{
sqlite3 *db;
sqlite3_stmt *stmt;
int rc;
rc = sqlite3_open("robotgowest.db", &db);
rc = sqlite3_prepare_v2(db, "INSERT INTO CaptorMetadata(CaptorNumber,TrialId) VALUES ( :captorNumber , :trialId)", -1, &stmt, NULL);
if (rc != SQLITE_OK)
return 1;
sqlite3_bind_int(stmt, 1, captorNumber);
sqlite3_bind_int(stmt, 2, trialId);
rc = sqlite3_step(stmt);
rc = sqlite3_finalize(stmt);
sqlite3_close(db);
return 0;
}
int insertCaptorDataElement(int trialId, int captorNumber, double lux, double mean, double power)
{
char *tableName = generateDataTableName(trialId, captorNumber);
sqlite3 *db;
sqlite3_stmt *stmt;
int rc;
rc = sqlite3_open("robotgowest.db", &db);
char sql[70] = "INSERT INTO ";
strcat(sql, tableName);
strcat(sql, "(Lux,Mean,Power) VALUES ( :lux , :mean , :power)");
rc = sqlite3_prepare_v2(db, sql, -1, &stmt, NULL);
if (rc != SQLITE_OK)
return 1;
sqlite3_bind_double(stmt, 1, lux);
sqlite3_bind_double(stmt, 2, mean);
sqlite3_bind_double(stmt, 3, power);
rc = sqlite3_step(stmt);
rc = sqlite3_finalize(stmt);
sqlite3_close(db);
return 0;
}
int createDb()
{
char *dbPosition = "./robotgowest.db";
if (access(dbPosition, F_OK))
{
initializeTrialsTable();
initializeCaptorMetadataTable();
return 0;
}
else
{
printf("Database already initialized");
return 1;
}
}
int getMaxTrialIdRow()
{
int rc;
sqlite3 *db;
sqlite3_stmt *stmt;
int data = -1;
rc = sqlite3_open("robotgowest.db", &db);
if (rc != SQLITE_OK)
{
return -1;
}
rc = sqlite3_prepare_v2(db, "SELECT MAX(Id) FROM Trials", -1, &stmt, NULL);
if (sqlite3_step(stmt) == SQLITE_ROW)
{
data = sqlite3_column_int(stmt, 0);
}
rc = sqlite3_finalize(stmt);
sqlite3_close(db);
return data;
}
int initiaizeNewTrial(char *position, double frequency, bool *captorOneHot)
{
insertTrailElement(position, -1, frequency);
int trialId = getMaxTrialIdRow();
for (int i = 0; i < 8; i++)
{
if (captorOneHot[i])
{
insertCaptorMetadataElement(i + 1, trialId);
createCaptorDataTable(trialId, i + 1);
}
}
return 0;
}

View file

@ -1,24 +1,29 @@
#include "fileGestion.h" #include "./Include/fileGestion.h"
/** /**
* @brief function that delete nRow lign in the beginning of the file rawData.csv . This function is necessary to not deal with the same ligns over and over * @brief function that delete nRow lign in the beginning of the file rawData.csv . This function is necessary to not deal with the same ligns over and over
* *
* @param nRow number of lign in the beginning of the file rawData.csv that has to be delete * @param nRow number of lign in the beginning of the file rawData.csv that has to be delete
*/ */
void clearRawData(int nRow){ void clearRawData(int nRow)
{
char buffer[256]; char buffer[256];
FILE *f = fopen("newFile.csv","w+"); FILE *f = fopen("newFile.csv", "w+");
FILE *g = fopen("rawData.csv","r"); FILE *g = fopen("rawData.csv", "r");
for(int i = 0; i < nRow; i++){ //first the program read the first nRow ligns of the csv file but do nothing for (int i = 0; i < nRow; i++)
fgets(buffer , sizeof buffer , g); { // first the program read the first nRow ligns of the csv file but do nothing
fgets(buffer, sizeof buffer, g);
} }
while(1){ //then, till the end of the csv file it copy the lign to a new csv : newFile.csv while (1)
if(!fgets(buffer,sizeof buffer , g)) break; { // then, till the end of the csv file it copy the lign to a new csv : newFile.csv
fprintf(f,"%s",buffer); if (!fgets(buffer, sizeof buffer, g))
break;
fprintf(f, "%s", buffer);
} }
remove("rawData.csv"); rename("newFile.csv", "rawData.csv"); //finally we remove the original file and rename the new one to replace rawData.csv remove("rawData.csv");
fclose(f); fclose(g); rename("newFile.csv", "rawData.csv"); // finally we remove the original file and rename the new one to replace rawData.csv
fclose(f);
fclose(g);
} }
/** /**
* @brief use to write one lign in the file "fileName" * @brief use to write one lign in the file "fileName"
@ -26,15 +31,19 @@ void clearRawData(int nRow){
* @param array array that contaign all the values to write in the file * @param array array that contaign all the values to write in the file
* @param nCol size of the array (correspond to the number of captor used) * @param nCol size of the array (correspond to the number of captor used)
*/ */
void writeDataInFile(char* fileName , double array[], int nCol){ void appendDataInFile(char *fileName, double array[], int nCol)
FILE *f = fopen(fileName,"a+"); {
for(int i = 0 ; i < nCol ; i++){ FILE *f = fopen(fileName, "a+");
if( i < nCol-1){ for (int i = 0; i < nCol; i++)
{
if (i < nCol - 1)
{
fprintf(f, "%f , ", array[i]); fprintf(f, "%f , ", array[i]);
} else { }
else
{
fprintf(f, "%f\n", array[i]); fprintf(f, "%f\n", array[i]);
} }
} }
fclose(f); fclose(f);
} }

View file

@ -1,48 +1,165 @@
#include "getArray.h" #include "./Include/getArray.h"
#include "fileGestion.h" #include "./Include/fileGestion.h"
//#include <string.h> #include "./Include/initialParameters.h"
// #include <string.h>
long **get(int N, int M) /* Allocate the array */ long **getlongArray(int N, int M) /* Allocate the array */
{ {
/* Check if allocation succeeded. (check for NULL pointer) */ /* Check if allocation succeeded. (check for NULL pointer) */
int i; int i;
long **array; long **array;
array = (long **) malloc(N*sizeof(long *)); array = (long **)malloc(N * sizeof(long *));
for(i = 0 ; i < N ; i++) for (i = 0; i < N; i++)
array[i] = (long *) malloc( M*sizeof(long) ); array[i] = (long *)malloc(M * sizeof(long));
return array; return array;
} }
void fillArrayWithRawData(char *rawDataFileName,long** p, int N, int M) { double **getDoubleArray(int N, int M) /* Allocate the array */
int i, j; {
char *buffer; /* Check if allocation succeeded. (check for NULL pointer) */
size_t bufsize = 200; int i;
buffer = (char *)malloc(bufsize * sizeof(char)); double **array;
char* token; array = (double **)malloc(N * sizeof(double *));
for (i = 0; i < N; i++)
FILE *f = fopen(rawDataFileName,"r"); array[i] = (double *)malloc(M * sizeof(double));
return array;
for(i = 0 ; i < N ; i++){
if (!getline(&buffer, &bufsize, f)) break; // condition d'arret de la boucle si fichier fini
j = 0;
while((token = strsep(&buffer,",")) != NULL){ // séparation valeur par virgule initiale : csv
p[i][j] = atoi(token);
j++;
}
}
fclose(f);
} }
void fillArrayWithRawData(char *rawDataFileName, long **p, int N, int M)
{
int i = 0;
// char *buffer;
// size_t bufsize = 200;
// buffer = (char *)malloc(bufsize * sizeof(char));
// char *token;
FILE *f = fopen(rawDataFileName, "r");
long t, c1, c2, c3, c4, c5, c6, c7, c8;
switch (M)
{
case 2:
while (fscanf(f, "%ld,%ld[^\n] ", &t, &c1) != EOF)
{
p[i][0] = t;
p[i][1] = c1;
i++;
}
break;
case 3:
while (fscanf(f, "%ld,%ld,%ld[^\n] ", &t, &c1, &c2) != EOF)
{
p[i][0] = t;
p[i][1] = c1;
p[i][2] = c2;
i++;
}
break;
case 4:
while (fscanf(f, "%ld,%ld,%ld,%ld[^\n] ", &t, &c1, &c2, &c3) != EOF)
{
p[i][0] = t;
p[i][1] = c1;
p[i][2] = c2;
p[i][3] = c3;
i++;
}
break;
case 5:
while (fscanf(f, "%ld,%ld,%ld,%ld,%ld[^\n] ", &t, &c1, &c2, &c3, &c4) != EOF)
{
p[i][0] = t;
p[i][1] = c1;
p[i][2] = c2;
p[i][3] = c3;
p[i][4] = c4;
i++;
}
break;
case 6:
while (fscanf(f, "%ld,%ld,%ld,%ld,%ld,%ld[^\n] ", &t, &c1, &c2, &c3, &c4, &c5) != EOF)
{
p[i][0] = t;
p[i][1] = c1;
p[i][2] = c2;
p[i][3] = c3;
p[i][4] = c4;
p[i][5] = c5;
i++;
}
break;
case 7:
while (fscanf(f, "%ld,%ld,%ld,%ld,%ld,%ld,%ld[^\n] ", &t, &c1, &c2, &c3, &c4, &c5, &c6) != EOF)
{
p[i][0] = t;
p[i][1] = c1;
p[i][2] = c2;
p[i][3] = c3;
p[i][4] = c4;
p[i][5] = c5;
p[i][6] = c6;
i++;
}
break;
case 8:
while (fscanf(f, "%ld,%ld,%ld,%ld,%ld,%ld,%ld,%ld[^\n] ", &t, &c1, &c2, &c3, &c4, &c5, &c6, &c7) != EOF)
{
p[i][0] = t;
p[i][1] = c1;
p[i][2] = c2;
p[i][3] = c3;
p[i][4] = c4;
p[i][5] = c5;
p[i][6] = c6;
p[i][7] = c7;
i++;
}
break;
case 9:
while (fscanf(f, "%ld,%ld,%ld,%ld,%ld,%ld,%ld,%ld,%ld[^\n] ", &t, &c1, &c2, &c3, &c4, &c5, &c6, &c7, &c8) != EOF)
{
p[i][0] = t;
p[i][1] = c1;
p[i][2] = c2;
p[i][3] = c3;
p[i][4] = c4;
p[i][5] = c5;
p[i][6] = c6;
p[i][7] = c7;
p[i][8] = c8;
i++;
}
break;
default:
printf("bad column size -> time + nbr captor\n");
}
/************** Debug part **************/
// for (int ii = 0; ii < nRowRawData; ii++)
// {
// for (int y = 0; y < nCol; y++)
// {
// printf("%ld,", p[ii][y]);
// if (y == nCol - 1)
// printf("lol\n");
// }
// }
fclose(f);
}
/** /**
* @brief print all the element of a bidimensionnal array p of shape : N x M * @brief print all the element of a bidimensionnal array p of shape : N x M
*/ */
void printArrayData(long** p, int N, int M) { void printArrayData(long **p, int N, int M)
int i, j; {
for(i = 0 ; i < N ; i++){ for (int i = 0; i < N; i++)
printf("line n°%d : " , i); {
for(int j = 0 ; j < M ; j++){ printf("line n°%d : ", i);
printf("%d , " , p[i][j]); for (int j = 0; j < M; j++)
if(j==(M-1)) printf("\n"); {
printf("%ld , ", p[i][j]);
if (j == (M - 1))
printf("\n");
} }
} }
} }
@ -55,9 +172,14 @@ void printArrayData(long** p, int N, int M) {
* @return true if the array contaign no NULL element * @return true if the array contaign no NULL element
* @return false if at least one element is null * @return false if at least one element is null
*/ */
bool checkArrayFullyFill(long **p, int N , int M){ bool checkArrayFullyFill(long **p, int N)
for(int i = 0 ; i < N ; i++){ {
if(p[i][0] == '\0'){ return false; } for (int i = 0; i < N; i++)
{
if (p[i][0] == '\0')
{
return false;
}
} }
return true; return true;
} }
@ -68,10 +190,12 @@ bool checkArrayFullyFill(long **p, int N , int M){
* @param p array to free memory * @param p array to free memory
* @param N number of rows in array * @param N number of rows in array
*/ */
void freeArray(long **p, int N) { void freeArray(long **p, int N)
int i; {
for(i = 0 ; i < N ; i++) for (int i = 0; i < N; i++)
{
free(p[i]); free(p[i]);
}
free(p); free(p);
} }
@ -83,15 +207,16 @@ void freeArray(long **p, int N) {
* @param M numbers of columns to have i the array * @param M numbers of columns to have i the array
* @return long** the array fill with raw data * @return long** the array fill with raw data
*/ */
long **getRawDataArray(char* rawDataFileName , int N , int M){ long **getRawDataArray(char *rawDataFileName)
{
long **p; long **p;
p = get(N, M); p = getlongArray(nRowRawData, nCol);
fillArrayWithRawData(rawDataFileName,p ,N, M); fillArrayWithRawData(rawDataFileName, p, nRowRawData, nCol);
if(checkArrayFullyFill(p,N,M)){ // if(checkArrayFullyFill(p,nRow)){
//clearRawData(N); // clearRawData(N);
return p; return p;
} /*}
else{ else{
return NULL; return NULL;
} }*/
} }

View file

@ -1,11 +0,0 @@
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <stdlib.h>
#include <stdbool.h>
#include <inttypes.h>
long **getRawDataArray(char *rawDataFileName, int N , int M);
void printArrayData(long** p, int N, int M);
void freeArray(long **p, int N);
bool checkArrayFullyFill(long **p, int N , int M);

42
Code-C/growthRate.c Normal file
View file

@ -0,0 +1,42 @@
#include "./Include/initialParameters.h"
#include "./Include/fileGestion.h"
#include "./Include/getArray.h"
#include "./Include/growthRate.h"
/**
* @brief calculate de growth rate between to point next to each other
*
* @param dataArray array with data of points next to each other for each captor (dim : 2 * nCols)
* @param gRateArray array that contaigns results of the growth ratecalculation
*/
void growthRateCalculation(double **dataArray, double *gRateArray)
{
for (int i = 0; i < nCol - 1; i++)
{
gRateArray[i] = (dataArray[0][i] - dataArray[1][i]) / period;
}
}
/**
* @brief calcul then print differentiate in csv
*
* @param dataLign
*/
void growthRateFunction(double **dataLign, char *fileName)
{
double gRateArray[nCol - 1];
growthRateCalculation(dataLign, gRateArray);
appendDataInFile(fileName, gRateArray, nCol - 1);
}
void sumColArray(long **p, double res[], int N, int M)
{
for (int i = 1; i < M; i++)
{
int j = 0;
res[i] = 0;
while (j < N)
{
res[i] += p[i][j];
j++;
}
}
}

Binary file not shown.

View file

@ -1,80 +1,186 @@
#include <pthread.h> #include <pthread.h>
#include <stdbool.h> #include <stdbool.h>
#include "simulateFlux.h" #include <sqlite3.h>
#include "power.h"
#include "initialParameters.h" #include "./Include/simulateFlux.h"
#include "queue.h" #include "./Include/power.h"
#include "average.h" #include "./Include/initialParameters.h"
#include "./Include/queue.h"
#include "./Include/average.h"
#include "./Include/growthRate.h"
#include "./Include/getArray.h"
#include "./Include/database.h"
bool rawDataWriteFlag; bool rawDataWriteFlag;
int nRow = 500; int nRowRawData = 5000;
int nCol = 9; int nRowGR = 150;
int nCol = 1;
double freqEch = 250; double freqEch = 250;
Pqueue firstRawDataQueue; int nbRowBinFile = 900011;
int nbRowIgnore = 19;
int selectionCaptors[] = {1,2,3,4,5,6,7,8}; Pqueue firstRawDataQueue;
int sizeSelectionArray = 8; // Captor 1 2 3 4 5 6 7 8
bool selectionCaptors[] = {true, false, true, false, false, false, true, false};
int cptData = 0; int cptData = 0;
int cptFile = 1; int cptFile = 1;
int cptValue = 0;
double period = 0; double period = 0;
double invTimeBandWidth = 0; double invTimeBandWidth = 0;
void *threadCalculPower(void *vargp){ void *threadCalculPower(void *vargp)
{
Pqueue rawDataQueue = firstRawDataQueue; Pqueue rawDataQueue = firstRawDataQueue;
char* fileName; char *fileName;
while(rawDataWriteFlag){ while (rawDataWriteFlag)
while(queueGetNextE(rawDataQueue) != NULL){ {
while (queueGetNextE(rawDataQueue) != NULL)
{
rawDataQueue = queueGetNextE(rawDataQueue); rawDataQueue = queueGetNextE(rawDataQueue);
fileName = queueGetTabChar(rawDataQueue); fileName = queueGetTabChar(rawDataQueue);
power(fileName,nRow,nCol,period,invTimeBandWidth); powerFunction(fileName, NULL);
remove(fileName); remove(fileName);
} }
} }
return NULL;
} }
void *threadCalculAverage(void *vargp){ void *threadCalculAverage(void *vargp)
{
Pqueue rawDataQueue = firstRawDataQueue; Pqueue rawDataQueue = firstRawDataQueue;
char* fileName; char *fileName;
while(rawDataWriteFlag){
while(queueGetNextE(rawDataQueue) != NULL){ while (rawDataWriteFlag)
{
while (queueGetNextE(rawDataQueue) != NULL)
{
rawDataQueue = queueGetNextE(rawDataQueue); rawDataQueue = queueGetNextE(rawDataQueue);
fileName = queueGetTabChar(rawDataQueue); fileName = queueGetTabChar(rawDataQueue);
average(fileName,nRow,nCol); printf("%s\n", fileName);
averageFunction(fileName, NULL);
remove(fileName); remove(fileName);
} }
} }
return NULL;
} }
void *threadCalculBoth(void *vargp){ void *threadCalculBoth(void *vargp)
{
Pqueue rawDataQueue = firstRawDataQueue; Pqueue rawDataQueue = firstRawDataQueue;
char* fileName; char *fileName;
while(rawDataWriteFlag){ while (rawDataWriteFlag)
while(queueGetNextE(rawDataQueue) != NULL){ {
while (queueGetNextE(rawDataQueue) != NULL)
{
rawDataQueue = queueGetNextE(rawDataQueue); rawDataQueue = queueGetNextE(rawDataQueue);
fileName = queueGetTabChar(rawDataQueue); fileName = queueGetTabChar(rawDataQueue);
power(fileName,nRow,nCol,period,invTimeBandWidth); powerFunction(fileName, NULL);
average(fileName,nRow,nCol); averageFunction(fileName, NULL);
remove(fileName); remove(fileName);
} }
} }
return NULL;
} }
int main(int argc , char** argv){ void *threadCalculGrowthRate(void *vargp)
{
Pqueue rawDataQueue = firstRawDataQueue;
char *fileName;
// double pw[nCol-1];
double **dataLignPw = getDoubleArray(2, nCol);
double **dataLignAv = getDoubleArray(2, nCol);
int i = 0;
while (rawDataWriteFlag)
{
while (queueGetNextE(rawDataQueue) != NULL)
{
rawDataQueue = queueGetNextE(rawDataQueue);
fileName = queueGetTabChar(rawDataQueue);
printf("%s\n", fileName);
if (i < 2)
{
if (i == 1)
{
powerFunction(fileName, dataLignPw);
averageFunction(fileName, dataLignAv);
growthRateFunction(dataLignPw, "growthRatePw.csv");
growthRateFunction(dataLignAv, "growthRateAv.csv");
}
else
{
averageFunction(fileName, dataLignAv);
powerFunction(fileName, dataLignPw);
}
i++;
}
else
{
for (int y = 0; y < (nCol - 1); y++)
{
dataLignPw[0][y] = dataLignPw[1][y];
dataLignAv[0][y] = dataLignAv[1][y];
}
powerFunction(fileName, dataLignPw);
averageFunction(fileName, dataLignAv);
growthRateFunction(dataLignPw, "growthRatePw.csv");
growthRateFunction(dataLignPw, "growthRateAv.csv");
}
remove(fileName);
}
}
return NULL;
}
int main(int argc, char **argv)
{
for (int i = 0; i < 8; i++)
{
if (selectionCaptors[i])
{
nCol++;
}
}
printf("%d", nCol);
rawDataWriteFlag = true; rawDataWriteFlag = true;
period = 1 / freqEch; period = 1 / freqEch;
invTimeBandWidth = 1 /(nRow * period); invTimeBandWidth = 1 / (nRowRawData * period);
firstRawDataQueue = queueCreateEmpty(); // change this for create empty firstRawDataQueue = queueCreateEmpty(); // change this for create empty
pthread_t rawData; // launch DB related function
pthread_create(&rawData , NULL, threadSimulateFlux, (void *)&rawData); sqlite3_initialize();
createDb(); // Created the Db if not exist
initiaizeNewTrial("test", freqEch, selectionCaptors); // Initialize Trials data and tables
pthread_t calcul; // pthread_t rawData;
pthread_create(&calcul , NULL, threadCalculBoth, (void *)&calcul); // if (pthread_create(&rawData, NULL, threadSimulateFlux, "threadSimulflux") != 0)
// {
// perror("threadSimulflux() error");
// exit(1);
// }
pthread_exit(NULL); // pthread_t calculAverage;
// if (pthread_create(&calculAverage, NULL, threadCalculAverage, "threadCalculAverage"))
// {
// perror("threadCalculAverage() error");
// exit(1);
// }
// pthread_t calculGrowthRate;
// if (pthread_create(&calculGrowthRate, NULL, threadCalculGrowthRate, "threadCalculGrowthRate"))
// {
// perror("threadcalculGrowthRate() error");
// exit(1);
// }
// pthread_exit(NULL);
sqlite3_shutdown();
} }

View file

@ -1,29 +1,30 @@
#include "power.h" #include <math.h>
#include "getArray.h"
#include "fileGestion.h" #include "./Include/power.h"
#include "initialParameters.h" #include "./Include/getArray.h"
#include "queue.h" #include "./Include/fileGestion.h"
#include "./Include/initialParameters.h"
#include "./Include/queue.h"
/** /**
* @brief realize the power calcul * @brief realize the powerThreadFunction calcul
* *
* @param p array with all the values that will be used for the calcul * @param p array with all the values that will be used for the calcul
* @param powerArray array where results are stocked * @param powerArray array where results are stocked
* @param N number of rows in p
* @param M number of columns in p
*/ */
void powerCalculation(long **p, double powerArray[] , int N, int M , double period , double invTimeBandwidth){ void powerCalculation(long **p, double powerArray[])
for(int i = 0; i < M-1; i++){ {
for (int i = 0; i < nCol; i++)
{
int j = 0; int j = 0;
powerArray[i] = 0; powerArray[i] = 0;
while(j < N-1){
double aire = ( pow(p[j][i+1],2) + pow(p[j+1][i+1],2) ) / 2 * period; while (j < nRowRawData)
//printf("aire [%d,%d] : %f\n",j,i,aire); {
powerArray[i] += aire; powerArray[i] += pow(p[j][i + 1], 2);
j++; j++;
} }
powerArray[i] *= invTimeBandwidth; powerArray[i] /= nRowRawData;
//printf("%f\n", powerArray[i]);
} }
} }
@ -34,16 +35,23 @@ void powerCalculation(long **p, double powerArray[] , int N, int M , double peri
* @param N number of rows in the file * @param N number of rows in the file
* @param M number of columns in the file * @param M number of columns in the file
*/ */
bool power(char* rawDataFileName,int N , int M, double periode , double invTimeBandwidth){ void powerFunction(char *rawDataFileName, double **pw)
long **p = getRawDataArray(rawDataFileName,N, M); {
double pw[8]; long **p = getRawDataArray(rawDataFileName);
if(p !=NULL){ // printArrayData(p, nRowRawData, nCol);
powerCalculation(p,pw,N,M,periode,invTimeBandwidth); double pww[nCol - 1];
writeDataInFile("powerData.csv",pw,8); if (p != NULL)
freeArray(p,N); {
return true; if (pw == NULL)
{
powerCalculation(p, pww);
appendDataInFile("powerData.csv", pww, nCol - 1);
} }
else{ else
return false; {
powerCalculation(p, pw[1]);
appendDataInFile("powerData.csv", pw[1], nCol - 1);
}
freeArray(p, nRowRawData);
} }
} }

View file

@ -1,4 +0,0 @@
#include <math.h>
#include <stdbool.h>
bool power(char* rawDataFileName,int N , int M, double periode , double invTimeBandwidth);

View file

@ -2,7 +2,9 @@
#include <stdio.h> #include <stdio.h>
#include <stdlib.h> #include <stdlib.h>
#include <string.h> #include <string.h>
#include "queue.h" #include <stdbool.h>
#include "./Include/queue.h"
/** /**
* @brief struct queue struct used for queueing string name * @brief struct queue struct used for queueing string name
@ -11,9 +13,10 @@
* @param tabChar char array pointer * @param tabChar char array pointer
* @param pNextE point next element * @param pNextE point next element
*/ */
struct queue { struct queue
{
int charLen; int charLen;
char* tabChar; char *tabChar;
Pqueue pNextE; Pqueue pNextE;
}; };
@ -24,8 +27,9 @@ typedef struct queue *Pqueue;
* *
* @return Pqueue * @return Pqueue
*/ */
Pqueue queueCreateEmpty(){ Pqueue queueCreateEmpty()
Pqueue new = (Pqueue) malloc(sizeof(struct queue)); {
Pqueue new = (Pqueue)malloc(sizeof(struct queue));
assert(new); assert(new);
new->charLen = 0; new->charLen = 0;
new->tabChar = NULL; new->tabChar = NULL;
@ -33,7 +37,7 @@ Pqueue queueCreateEmpty(){
return new; return new;
} }
/************ SETTER ************/ /************ SETTER ************/
/** /**
* @brief set char size array * @brief set char size array
@ -41,7 +45,8 @@ Pqueue queueCreateEmpty(){
* @param elem targeted element * @param elem targeted element
* @param _charLen size of char array * @param _charLen size of char array
*/ */
void queueSetCharLen(Pqueue elem, int _charLen){ void queueSetCharLen(Pqueue elem, int _charLen)
{
assert(elem); assert(elem);
elem->charLen = _charLen; elem->charLen = _charLen;
} }
@ -53,12 +58,14 @@ void queueSetCharLen(Pqueue elem, int _charLen){
* @param _charLen char array size * @param _charLen char array size
* @param _tabChar pointer to static char array * @param _tabChar pointer to static char array
*/ */
void queueSetTabChar(Pqueue elem, int _charLen, const char *_tabChar){ void queueSetTabChar(Pqueue elem, int _charLen, const char *_tabChar)
{
assert(elem); assert(elem);
assert(_tabChar); assert(_tabChar);
elem->charLen = _charLen; elem->charLen = _charLen;
elem->tabChar = calloc(_charLen , sizeof(char)); elem->tabChar = calloc(_charLen, sizeof(char));
for(int i = 0; i < _charLen; i++){ for (int i = 0; i < _charLen; i++)
{
elem->tabChar[i] = _tabChar[i]; elem->tabChar[i] = _tabChar[i];
} }
} }
@ -68,13 +75,14 @@ void queueSetTabChar(Pqueue elem, int _charLen, const char *_tabChar){
* @param elem target element * @param elem target element
* @param next next element * @param next next element
*/ */
void queueSetNextE(Pqueue elem , Pqueue next){ void queueSetNextE(Pqueue elem, Pqueue next)
{
assert(elem); assert(elem);
assert(next); assert(next);
elem -> pNextE = next; elem->pNextE = next;
} }
/************ GETTER ************/ /************ GETTER ************/
/** /**
* @brief switch to the next element * @brief switch to the next element
@ -82,7 +90,8 @@ void queueSetNextE(Pqueue elem , Pqueue next){
* @param elem current element * @param elem current element
* @return Pqueue next element * @return Pqueue next element
*/ */
Pqueue queueGetNextE(Pqueue elem){ Pqueue queueGetNextE(Pqueue elem)
{
assert(elem); assert(elem);
return elem->pNextE; return elem->pNextE;
} }
@ -93,7 +102,8 @@ Pqueue queueGetNextE(Pqueue elem){
* @param elem targeted element * @param elem targeted element
* @return int * @return int
*/ */
int queueGetCharLen(Pqueue elem){ int queueGetCharLen(Pqueue elem)
{
assert(elem); assert(elem);
return elem->charLen; return elem->charLen;
} }
@ -104,12 +114,13 @@ int queueGetCharLen(Pqueue elem){
* @param elem targeted elemnt * @param elem targeted elemnt
* @return char* * @return char*
*/ */
char * queueGetTabChar(Pqueue elem){ char *queueGetTabChar(Pqueue elem)
{
assert(elem); assert(elem);
return elem->tabChar; return elem->tabChar;
} }
/************ ************/ /************ ************/
/** /**
* @brief Create Element of queue * @brief Create Element of queue
@ -117,8 +128,9 @@ char * queueGetTabChar(Pqueue elem){
* @param lenChar size of char array that will be created * @param lenChar size of char array that will be created
* @return Pqueue new element created * @return Pqueue new element created
*/ */
Pqueue queueCreateE(int lenChar, const char* _tabChar){ Pqueue queueCreateE(int lenChar, const char *_tabChar)
Pqueue new = (Pqueue) malloc(sizeof(struct queue)); {
Pqueue new = (Pqueue)malloc(sizeof(struct queue));
assert(new); assert(new);
queueSetTabChar(new, lenChar, _tabChar); queueSetTabChar(new, lenChar, _tabChar);
new->pNextE = NULL; new->pNextE = NULL;
@ -131,16 +143,18 @@ Pqueue queueCreateE(int lenChar, const char* _tabChar){
* @param elem current element * @param elem current element
* @return Pqueue current element * @return Pqueue current element
*/ */
Pqueue queueRmLastE(Pqueue elem){ Pqueue queueRmLastE(Pqueue elem)
{
assert(elem); assert(elem);
Pqueue tmp = elem, previous = NULL; Pqueue tmp = elem, current = NULL;
while(elem->pNextE != NULL){ while (tmp->pNextE != NULL)
previous = tmp; {
current = tmp;
tmp = queueGetNextE(tmp); tmp = queueGetNextE(tmp);
} }
free(tmp->tabChar); free(tmp->tabChar);
free(tmp); free(tmp);
previous->pNextE = NULL; current->pNextE = NULL;
return elem; return elem;
} }
@ -148,11 +162,18 @@ Pqueue queueRmLastE(Pqueue elem){
* @brief remove and free the first element of queue * @brief remove and free the first element of queue
* *
* @param elem target to remove (the element need to be the first) * @param elem target to remove (the element need to be the first)
* @return next element of the queue
*/ */
void queueRmFrstE(Pqueue elem){ Pqueue queueRmFrstE(Pqueue elem)
{
assert(elem); assert(elem);
Pqueue next = queueGetNextE(elem);
if (elem->tabChar != NULL)
{
free(elem->tabChar); free(elem->tabChar);
}
free(elem); free(elem);
return next;
} }
/** /**
@ -161,7 +182,8 @@ void queueRmFrstE(Pqueue elem){
* @param elem * @param elem
* @return Pqueue * @return Pqueue
*/ */
Pqueue queueNextDelFrst(Pqueue elem){ Pqueue queueNextDelFrst(Pqueue elem)
{
assert(elem); assert(elem);
Pqueue tmp = elem->pNextE; Pqueue tmp = elem->pNextE;
queueRmFrstE(elem); queueRmFrstE(elem);
@ -175,17 +197,34 @@ Pqueue queueNextDelFrst(Pqueue elem){
* @param str the string will be bind at the element * @param str the string will be bind at the element
* @param len the lenght of char array * @param len the lenght of char array
*/ */
void queueAddLastQ(Pqueue elem, const char* str, int len){ void queueAddLastQ(Pqueue elem, const char *str, int len)
{
assert(elem); assert(elem);
assert(str); bool str1 = true;
Pqueue next = elem, previous = NULL; if (len == 0)
while(next->pNextE != NULL){ {
previous = next; str = false;
next = queueGetNextE(next);
} }
previous = next; else
next = queueCreateE(len,str); {
queueSetNextE(previous,next); assert(str);
}
Pqueue current = elem, next = NULL;
while (current->pNextE != NULL)
{
current = queueGetNextE(current);
}
if (str1 == true)
{
next = queueCreateE(len, str);
}
else
{
next = queueCreateEmpty();
}
queueSetNextE(current, next);
} }
/** /**
@ -193,13 +232,16 @@ void queueAddLastQ(Pqueue elem, const char* str, int len){
* *
* @param elem targeted element * @param elem targeted element
*/ */
void queuePrintE(Pqueue elem){ void queuePrintE(Pqueue elem)
{
Pqueue next = queueGetNextE(elem); Pqueue next = queueGetNextE(elem);
printf("\nFile Name : %s \n(size of the file name : %d)\n",elem -> tabChar , elem -> charLen ); printf("\nFile Name : %s \n(size of the file name : %d)\n", elem->tabChar, elem->charLen);
if(next != NULL){ if (next != NULL)
printf("Next File : %s\n", next ->tabChar); {
printf("Next File : %s\n", next->tabChar);
} }
else{ else
{
printf("No nextFile existing (null)\n"); printf("No nextFile existing (null)\n");
} }
} }
@ -209,12 +251,25 @@ void queuePrintE(Pqueue elem){
* *
* @param firstE * @param firstE
*/ */
void queuePrintWholeQ(Pqueue firstE){ void queuePrintWholeQ(Pqueue firstE)
{
queuePrintE(firstE); queuePrintE(firstE);
Pqueue elem = firstE; Pqueue elem = firstE;
while(queueGetNextE(elem) != NULL){ while (queueGetNextE(elem) != NULL)
{
elem = queueGetNextE(elem); elem = queueGetNextE(elem);
queuePrintE(elem); queuePrintE(elem);
} }
} }
void queueDelAll(Pqueue elem)
{
assert(elem);
Pqueue condemned = elem, tmp;
while (condemned != NULL)
{
tmp = queueGetNextE(condemned);
queueRmFrstE(condemned);
condemned = tmp;
}
}

View file

@ -1,27 +0,0 @@
typedef struct queue *Pqueue;
//Constructors
Pqueue queueCreateE(int lenChar, const char* _tabChar);
Pqueue queueCreateEmpty();
//Getters
Pqueue queueGetNextE(Pqueue elem);
int queueGetCharLen(Pqueue elem);
char* queueGetTabChar(Pqueue elem);
//Setters
void queueSetCharLen(Pqueue elem, int _charLen);
void queueSetTabChar(Pqueue elem, int _charLen, const char *_tabChar);
void queueSetNextE(Pqueue elem , Pqueue next);
void queueAddLastQ(Pqueue elem, const char* str, int len);
//Removers
Pqueue queueRmLastE(Pqueue elem);
void queueRmFrstE(Pqueue elem);
Pqueue queueNextDelFrst(Pqueue elem);
//print function
void queuePrintE(Pqueue elem);
void queuePrintWholeQ(Pqueue firstE);

View file

@ -1,6 +0,0 @@
#!/bin/bash
cd ../RawDataFiles
for i in *
do
rm $i
done

BIN
Code-C/robotgowest.db Normal file

Binary file not shown.

View file

@ -1,6 +1,9 @@
#include "simulateFlux.h" #include "./Include/simulateFlux.h"
#include "initialParameters.h" #include "./Include/initialParameters.h"
#include "queue.h" #include "./Include/queue.h"
#include <time.h>
#include <errno.h>
/** /**
* @brief convert an interger N into a char* * @brief convert an interger N into a char*
@ -8,29 +11,32 @@
* @param N * @param N
* @return char* * @return char*
*/ */
char* convertIntegerToChar(int N) char *convertIntegerToChar(int N)
{ {
// Count digits in number N // Count digits in number N
int m = N; int m = N;
int digit = 0; int digit = 0;
while (m) { while (m)
{
digit++; digit++;
m /= 10; m /= 10;
} }
char* arr; char *arr;
char arr1[digit]; char arr1[digit];
arr = (char*)malloc(digit); arr = (char *)malloc(digit * sizeof(char));
int index = 0; int index = 0;
while (N) { while (N)
{
arr1[++index] = N % 10 + '0'; arr1[++index] = N % 10 + '0';
N /= 10; N /= 10;
} }
int i; int i;
for (i = 0; i < index; i++) { for (i = 0; i < index; i++)
{
arr[i] = arr1[index - i]; arr[i] = arr1[index - i];
} }
arr[i] = '\0'; arr[i] = '\0';
return (char*)arr; return (char *)arr;
} }
/** /**
@ -38,33 +44,43 @@ char* convertIntegerToChar(int N)
* *
* @return char* * @return char*
*/ */
char *createNewRawDataFileName(){ char *createNewRawDataFileName()
char *fileName = "../RawDataFiles/RawData"; {
char *fileName = "RawDataFiles/RawData";
char *extension = ".csv\0"; char *extension = ".csv\0";
char *fileNumber = convertIntegerToChar(cptFile); char *fileNumber = convertIntegerToChar(cptFile);
// char *fileNumber;
// sprintf(fileNumber, "%d", cptFile);
// printf("%s\n" , fileNumber);
char fileNameNumber[strlen(fileName)+strlen(fileNumber)]; char fileNameNumber[strlen(fileName) + strlen(fileNumber)];
char *fullFileName = malloc((strlen(fileNameNumber)+strlen(extension)) * sizeof(char*)); char *fullFileName = malloc((strlen(fileNameNumber) + strlen(extension)) * sizeof(char *));
strcpy( fileNameNumber, fileName ); strcpy(fileNameNumber, fileName);
strcat( fileNameNumber, fileNumber ); strcat(fileNameNumber, fileNumber);
strcpy( fullFileName, fileNameNumber ); strcpy(fullFileName, fileNameNumber);
strcat( fullFileName, extension ); strcat(fullFileName, extension);
return fullFileName; return fullFileName;
} }
int intInArray(int number , int *array , int N){ int intInArray(int number, int *array, int N)
for(int i = 0 ; i < N ; i++){ {
if(array[i] == number) return 0; for (int i = 0; i < N; i++)
{
if (array[i] == number)
return 0;
} }
return -1; return -1;
} }
int maxInArray(int *array , int N){ int maxInArray(int *array, int N)
{
int max = 0; int max = 0;
for(int i = 0 ; i < N ; i++){ for (int i = 0; i < N; i++)
if(array[i]>max) max = array[i]; {
if (array[i] > max)
max = array[i];
} }
return max; return max;
} }
@ -73,11 +89,25 @@ int maxInArray(int *array , int N){
* *
* @return int64_t * @return int64_t
*/ */
int64_t millis() int64_t millis()
{ {
struct timespec now; struct timespec now;
timespec_get(&now, TIME_UTC); timespec_get(&now, TIME_UTC);
return ((int64_t) now.tv_sec) * 1000 + ((int64_t) now.tv_nsec) / 1000000; return ((int64_t)((int64_t)now.tv_sec) * 1000 + ((int64_t)now.tv_nsec) / 1000000);
}
int lastIndexCaptor()
{
int lastIndex = 0;
for (int i = 1; i < 8; i++)
{
if (selectionCaptors[i])
{
lastIndex = i;
}
}
return lastIndex;
} }
/** /**
@ -87,70 +117,104 @@ int64_t millis()
* @return true if the lign is correctly write , else : * @return true if the lign is correctly write , else :
* @return false * @return false
*/ */
bool writeOneRawData(FILE *rawDataFile){ bool writeOneRawData(FILE *rawDataFile)
{
char buff[26]; char buff[26];
char buff2[18]; char buff2[18];
int32_t values[8]; int32_t values[8];
uint32_t valbin[8]; uint32_t valbin[8];
quartet value;
if (fread(&buff, 26, 1, stdin))
{
// printf("%d\n",cptValue);
if (cptValue < nbRowBinFile - nbRowIgnore)
{
FILE *timeFile = fopen("timeFile.csv", "a+");
fprintf(timeFile, "%ld\n", millis());
fclose(timeFile);
fprintf(rawDataFile, "%ld,", millis());
if(fread(&buff, 26, 1, stdin)) { if (strncmp(buff, "#################\n", (size_t)18) == 0)
fprintf(rawDataFile , "%d,", millis()); {
if (strncmp(buff, "#################\n", (size_t)18) == 0) { if (!(fread(&buff2, 18, 1, stdin)))
if (!(fread(&buff2, 18, 1, stdin))) { {
fprintf(stderr, "Erreur lecture après ###...#"); fprintf(stderr, "Erreur lecture après ###...#");
return 2; }
} else { else
{
strncpy(buff, &buff[18], 8); strncpy(buff, &buff[18], 8);
strncpy(&buff[8], buff2, 18); strncpy(&buff[8], buff2, 18);
} }
} }
int maxCApteurNb = maxInArray(selectionCaptors,sizeSelectionArray);
for (int i = 1; i < 9; i++){
if(intInArray(i,selectionCaptors,sizeSelectionArray)==0){
value.octet1 = buff[3*i+1];
value.octet2 = buff[3*i+2];
value.octet3 = buff[3*i+3];
value.octet4 = 0;
valbin[i] = buff[3*i+1]*256*256*256 + buff[3*i+2]*256*256 + buff[3*i+3]*256; int lastIndex = lastIndexCaptor();
for (int i = 1; i < 9; i++)
{
if (selectionCaptors[i - 1])
{
// quartet value;
// value.octet1 = buff[3 * i + 1];
// value.octet2 = buff[3 * i + 2];
// value.octet3 = buff[3 * i + 3];
// value.octet4 = 0;
valbin[i] = buff[3 * i + 1] * 256 * 256 * 256 + buff[3 * i + 2] * 256 * 256 + buff[3 * i + 3] * 256;
memcpy(&values[i], &valbin[i], sizeof(uint32_t)); memcpy(&values[i], &valbin[i], sizeof(uint32_t));
FILE *allRawDataFile = fopen("AllRawData.csv", "a+");
if (i - 1 == lastIndex)
if(i==maxCApteurNb){ {
fprintf(rawDataFile, "%d\n", values[i]/256); fprintf(rawDataFile, "%d\n", values[i] / 256);
fprintf(allRawDataFile, "%d\n", values[i] / 256);
} }
else{ else
fprintf(rawDataFile, "%d,", values[i]/256); {
fprintf(rawDataFile, "%d,", values[i] / 256);
fprintf(allRawDataFile, "%d,", values[i] / 256);
} }
fclose(allRawDataFile);
} }
} }
cptData++; cptData++;
//sleep(0.004); //simul freq here
/************** simul freq here **************/
// struct timespec ts;
// ts.tv_sec = 0;
// ts.tv_nsec = 4 * 1000000;
// nanosleep(&ts, &ts);
cptValue++;
return true; return true;
} }
else { else
{
return false; return false;
} }
}
return false;
} }
void *threadSimulateFlux(void *vargp){ void *threadSimulateFlux(void *vargp)
{
char *fileName = createNewRawDataFileName(); char *fileName = createNewRawDataFileName();
FILE *rawDataFile = fopen(fileName,"w+"); FILE *rawDataFile = fopen(fileName, "w+");
while(writeOneRawData(rawDataFile)){ while (writeOneRawData(rawDataFile))
if(cptData == nRow){ {
if (cptData == nRowRawData)
{
fclose(rawDataFile); fclose(rawDataFile);
cptData = 0; cptData = 0;
cptFile++; cptFile++;
//create struct here // create struct here
queueAddLastQ(firstRawDataQueue , fileName , strlen(fileName)); queueAddLastQ(firstRawDataQueue, fileName, strlen(fileName));
//prepare next file now // prepare next file now
fileName = createNewRawDataFileName(); fileName = createNewRawDataFileName();
FILE *rawDataFile = fopen(fileName,"w+"); rawDataFile = fopen(fileName, "w+");
// add test to get p then print it here // p is gotten from fileName file
} }
} }
rawDataWriteFlag = false; rawDataWriteFlag = false;
return NULL;
} }

View file

@ -0,0 +1,931 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h1>Forecast Project</h1>"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>Database</h2>"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<b>Import packages for DB</b>"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import yfinance as yfin\n",
"import sqlite3 as sql\n",
"import pandas as pd\n",
"import os\n",
"import datetime\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h3>YFianance API</h3>\n",
"<p>Function using the yfinance API to get all market data</p>"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<b>Tickers</b>"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"#array with all the studied currencies (yahoo finance currency names)\n",
"currencies = [\"BTC-USD\",\"ETH-USD\",\"TTE.PA\"] "
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"def getMetaData(s : str) -> dict :\n",
" \"\"\"Get the meta information for a Ticker (currency)\n",
" \n",
" Args : \n",
" s (str) : name of the Ticker\n",
" \n",
" Return : \n",
" dict : return all the meta information of the Ticker, the dict is the following : \n",
" - symbol : str ~> name online (yahoo finance) of the stock (is also the ticker name used by the yfinance API)\n",
" - instrumentType : str ~> type of stock, can be a cryptocurrency , a stock ...\n",
" - regularMarketTime int : ~> date when those informatation has been gathered in epoch/unix time \n",
" - gmtoffset int : ~> number of second of offset between the stock's market timezome and the gtm timzome\n",
" - timezone str : ~> name of the stock's market timezone\n",
" other var a disponnible but not used here (see yfinance documentation)\n",
" \"\"\" \n",
" ticker = yfin.Ticker(s)\n",
" ticker.history()\n",
" meta = ticker.history_metadata\n",
" return meta\n",
" \n",
"def getHistoryData(s : str) -> dict :\n",
" \"\"\"Get all the information since the creation of the Ticker (currency)\n",
" \n",
" Args : \n",
" s (str) : name of the Ticker\n",
" \n",
" Return : \n",
" dict : return all the information of the Ticker since creation, the dict contaign the following information : {Date,Open,Hight,Low,Close,Volume}\n",
" \"\"\"\n",
" ticker = yfin.Ticker(s)\n",
" data = ticker.history(\"max\", interval = '1d')\n",
" data['Date'] = data.index\n",
" data = data[[\"Date\", \"Open\", \"High\", \"Low\", \"Close\", \"Volume\"]]\n",
" data.reset_index(drop=True, inplace=True)\n",
" return data\n",
"\n",
"def getPeriodData(s : str , period : str) -> dict :\n",
" \"\"\"Get the information of the Ticker for a certain period\n",
" \n",
" Args : \n",
" s (str) : name of the Ticker\n",
" period (str) : duration of the period , the information period get are ending to the current day and start a period back can take value like {1d , 5d , 1m , 1y , ...}\n",
" \n",
" Return : \n",
" dict : return all the information of the Ticker for the period, the dict contaign the following information : {Date,Open,Hight,Low,Close,Volume}\n",
" \"\"\"\n",
" ticker = yfin.Ticker(s)\n",
" data = ticker.history(period = period ,interval = '1d')\n",
" data['Date'] = data.index\n",
" data = data[[\"Date\", \"Open\", \"High\",\"Low\", \"Close\", \"Volume\"]]\n",
" data.reset_index(drop=True, inplace=True)\n",
" return data"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h3>DataBase Gestion</h3>"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h4>Currencies_Metadata table</h4>\n",
"<p>This table contaign all the usefull metadata of each currencies fetched. It contaigns : </p>\n",
"<ul><li><b>currencyName</b> ~> name online (yahoo finance) of the stock (is also the ticker name used by the yfinance API)</li>\n",
"<li><b>instrumentType</b> ~> type of stock, can be a cryptocurrency , a stock ...</li>\n",
"<li><b>regularMarketTime</b> ~> date when those informatation has been gathered in epoch/unix time </li>\n",
"<li><b>gtmOffset</b> ~> number of second of offset between the stock's market timezome and the gtm timezome</li>\n",
"<li><b>timezone</b> ~> name of the stock's market timezone</li></ul>\n",
"It is possible to \n",
"<ul><li>Create / Fill <b>table</b></li>\n",
"<li>Insert / Get / Modify / Delete / Print <b>table elements</b></li></ul>"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"def createCurrencyMetadataTable():\n",
" \"\"\"Create a new currency table \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
"\n",
" requestCreate = '''CREATE TABLE IF NOT EXISTS CURRENCIES_METADATA (\n",
" [currencyId] INTEGER PRIMARY KEY,\n",
" [currencyName] TEXT, \n",
" [instrumentType] TEXT,\n",
" [regularMarketTime] TEXT,\n",
" [gtmOffset] INTEGER,\n",
" [timezone] TEXT\n",
" )'''\n",
" #currencyName : str ~> name online (yahoo finance) of the stock (is also the ticker name used by the yfinance API)\n",
" #instrumentType : str ~> type of stock, can be a cryptocurrency , a stock ...\n",
" #regularMarketTime int : ~> date when those informatation has been gathered in epoch/unix time \n",
" #gtmOffset int : ~> number of second of offset between the stock's market timezome and the gtm timzome\n",
" #timezone str : ~> name of the stock's market timezone\n",
" c.execute(requestCreate) # create a new currency table\n",
" db.commit()\n",
"\n",
"def getCurrencyMetadataElement(currencyId : int) -> dict :\n",
" \"\"\"Get an element of the currency table by its id\n",
"\n",
" Args:\n",
" currencyId (int): id of the currency in the table\n",
"\n",
" Returns:\n",
" dict: return the element as a dict with all metadata of a currency\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" request = f'''SELECT * FROM CURRENCIES_METADATA WHERE currencyId='{currencyId}' '''\n",
" c.execute(request)\n",
" return list(c.fetchone())\n",
"\n",
"def getIdByNameCurrencyMetadata(currencyName: str) -> int :\n",
" \"\"\"Get an element id of the currency table by its name\n",
"\n",
" Args:\n",
" currencyName (str): name of the stock\n",
"\n",
" Returns:\n",
" int: return the element id\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" request = f'''SELECT currencyId FROM CURRENCIES_METADATA WHERE currencyName='{currencyName}' '''\n",
" c.execute(request)\n",
" return c.fetchone()[0]\n",
"\n",
"def printCurrencyMetadataElement(currencyId : int):\n",
" \"\"\"Print an element of the currency table by its id \n",
"\n",
" Args:\n",
" currencyId (int): id of the currency in the table \n",
" \"\"\"\n",
" metaData = getCurrencyMetadataElement(currencyId)\n",
" header = f\"-------- MetaData for {metaData[1]} --------\"\n",
" nHeader = len(header)\n",
" print(header)\n",
" print(f\"\\nName : {metaData[1]}\")\n",
" print(f\"ID : {metaData[0]}\")\n",
" print(f\"instrumentType : {metaData[2]}\")\n",
" print(f\"regularMarketTime : {metaData[3]}\")\n",
" print(f\"gtmOffset : {metaData[4]}\")\n",
" print(f\"timezone : {metaData[5]}\")\n",
" footer = \"\\n\"\n",
" for i in range(nHeader): footer += \"-\"\n",
" print(footer)\n",
"\n",
"def insertCurrencyMetadataElement(currency : str):\n",
" \"\"\"Insert a new element in the currency table\n",
"\n",
" Args:\n",
" currency (str): name of the currency to fetch its metadata with the yFinance API\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" metaData = getMetaData(currency)\n",
" request = f'''INSERT INTO CURRENCIES_METADATA (currencyName,instrumentType,regularMarketTime,gtmOffset,timezone)\n",
" VALUES (\n",
" '{metaData['symbol']}',\n",
" '{metaData['instrumentType']}',\n",
" '{metaData['regularMarketTime']}',\n",
" {metaData['gmtoffset']},\n",
" '{metaData['timezone']}')'''\n",
" c.execute(request)\n",
" db.commit()\n",
"\n",
"def modifyCurrencyMetadataElement(currencyId : int, newMetaData : dict):\n",
" \"\"\"Modify an element in the currency table, the element to modify must be enteriely rewrite \n",
"\n",
" Args:\n",
" currencyId (int): id of the currency to modify in the currency table\n",
" newMetaData (dict): dictionnary contaigning all the information of the currency even the ones not changed\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" #All the information MUST be given in the newMetaData dict even the id and the ones not changed\n",
" request = f'''UPDATE CURRENCIES_METADATA \n",
" SET \n",
" currencyId={newMetaData['currencyId']},\n",
" currencyName='{newMetaData['symbol']}', \n",
" instrumentType='{newMetaData['instrumentType']}', \n",
" regularMarketTime='{newMetaData['regularMarketTime']}',\n",
" gtmOffset={newMetaData['gtmOffset']},\n",
" timezone='{newMetaData['timezone']}' \n",
" WHERE currencyId = {currencyId}'''\n",
" c.execute(request)\n",
" db.commit()\n",
"\n",
"def deleteCurrencyMetadataElement(currencyId : int):\n",
" \"\"\"Delete an element from the currency table\n",
"\n",
" Args:\n",
" currencyId (int): id of the currency to delete from the currency table\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" request = f'''DELETE FROM CURRENCIES_METADATA\n",
" WHERE currencyId = {currencyId}'''\n",
" c.execute(request)\n",
" db.commit()\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h4>Others</h4>\n",
"<p>Usefull an necessary function for next parts </p>"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"def cleanNameForSQL(s : str) -> str:\n",
"\t\"\"\"Return a new string that dont contaign special char to use it as sql table name\n",
"\n",
"\tArgs:\n",
"\t\ts (str): string where to remove special char \n",
"\n",
"\tReturns:\n",
"\t\tstr: new string without special char nor spaces \n",
"\t\"\"\"\n",
"\tfor char in ['-','.','^',' ']:\n",
"\t\ts = s.replace(char,\"\")\n",
"\treturn s"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h4>Market Tables</h4>\n",
"There is one market table for each currency fetched. It contaigns :\n",
"<ul><li><b>Date</b> ~> date of the information present</li>\n",
"<li><b>Open</b> ~> value of the stock when the market open</li>\n",
"<li><b>High</b> ~> highest value reach by the stock during the market time</li> \n",
"<li><b>Low</b> ~> lowest value reach by the stock during the market time</li>\n",
"<li><b>Close</b> ~> value of the stock when the market close </li>\n",
"<li><b>Volume</b> ~> total of stock sells during the market time</li>\n",
"</ul>\n",
"It is possible to \n",
"<ul><li>Create / Fill <b>table</b></li>\n",
"<li>Insert / Get / Print <b>table elements</b></li></ul>"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"def createCurrencyMarketTable(currencyName : str):\n",
" \"\"\"Create a new table if it dont alredy exist to contaign all the market infomation of a currency\n",
"\n",
" Args:\n",
" currencyName (str): name of the currency for which a new table is created\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" tableName = cleanNameForSQL(currencyName)\n",
" #each lign in the table represent a new day and contaign :\n",
" # Date : date ~> date of the information present\n",
" # Open : float ~> value of the stock when the market open\n",
" # High : float ~> highest value reach by the stock during the market time \n",
" # Low : float ~> lowest value reach by the stock during the market time\n",
" # Close : float ~> value of the stock when the market close \n",
" # Volume : float ~> total of stock sells during the market time \n",
" createNewCurrencyTable = f'''CREATE TABLE IF NOT EXISTS {tableName}_MARKET \n",
" ([Id] INTEGER PRIMARY KEY AUTOINCREMENT,\n",
" [Date] TEXT,\n",
" [Open] NUMBER,\n",
" [High] NUMBER,\n",
" [Low] NUMBER,\n",
" [Close] NUMBER,\n",
" [Volume] INTEGER);'''\n",
" c.execute(createNewCurrencyTable)\n",
" db.commit()\n",
"\n",
"def fillCurrencyMarketTable(currency : str , period : str = None):\n",
" \"\"\"Fill a currency market table with its information\n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency , used to get the information with the API\n",
" period (str, optional): default value is none : all the information since create are fetched, can be defined to limit the period (for value value autorized see yfinance doc). Defaults to None.\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
"\n",
" tableName = cleanNameForSQL(currency)\n",
" requestTemplate = f'''INSERT INTO {tableName}_MARKET (Date,Open,High,Low,Close,Volume)\n",
" VALUES '''\n",
"\n",
" #verify if @period has been enter as parameter\n",
" if period : data = getPeriodData(currency , period) #if yes get only period data\n",
" else : data = getHistoryData(currency) # else get all data since creation\n",
"\n",
" #for each lign in data fetched, the request add them in the table\n",
" for i in range(len(data)):\n",
" line = data.iloc[i]\n",
" #round(line[2:5],13) #all number are rounded to the 13th decimal to avoid some float type python issues \n",
" request =requestTemplate + f'''('{line['Date']}',\n",
" {line['Open']},\n",
" {line['High']},\n",
" {line['Low']},\n",
" {line['Close']},\n",
" {line['Volume']});'''\n",
"\n",
" c.execute(request)\n",
" db.commit()\n",
"\n",
"def getCurrencyMarketElementById(currency : str , elementId : int ) -> dict :\n",
" \"\"\"Get an element from a currency market table by its id \n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency , used to get the information with the API\n",
" elementId (int): id of the element to get \n",
"\n",
" Returns:\n",
" dict: element of the table which id is elementId \n",
" \"\"\"\n",
" tableName = cleanNameForSQL(currency)\n",
" request = f'''SELECT * FROM {tableName}_MARKET WHERE Id = {elementId} '''\n",
"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" c.execute(request)\n",
" return c.fetchone()[0]\n",
"\n",
"def getAllCurrencyMarketElement(currency : str ) -> pd.DataFrame :\n",
" \"\"\"Get all elements from a currency market table by its id \n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency , used to get the information with the API\n",
"\n",
" Returns:\n",
" list: list of all elements of the table \n",
" \"\"\"\n",
" tableName = cleanNameForSQL(currency)\n",
" request = f'''SELECT Date,Open,High,Low,Close,Volume FROM {tableName}_MARKET'''\n",
"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" c.execute(request)\n",
" res = pd.DataFrame(c.fetchall(),columns=['Date','Open','High','Low','Close','Volume'])\n",
" # for i in range(len(res)):\n",
" # res[i] = list(res[i])\n",
" return res\n",
"\n",
"def getCloseCurrencyMarketElement(currency : str ) -> list :\n",
" \"\"\"Get close elements from a currency market table by its id \n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency , used to get the information with the API\n",
"\n",
" Returns:\n",
" list: list of all elements of the table \n",
" \"\"\"\n",
" tableName = cleanNameForSQL(currency)\n",
" request = f'''SELECT Close FROM {tableName}_MARKET'''\n",
"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" c.execute(request)\n",
" res = c.fetchall()\n",
" for i in range(len(res)):\n",
" res[i] = list(res[i])[0]\n",
" return res\n",
"\n",
"def getDateCurrencyMarketElement(currency : str ) -> list :\n",
" \"\"\"Get date elements from a currency market table by its id \n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency , used to get the information with the API\n",
"\n",
" Returns:\n",
" list: list of all elements of the table \n",
" \"\"\"\n",
" tableName = cleanNameForSQL(currency)\n",
" request = f'''SELECT Date FROM {tableName}_MARKET'''\n",
"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" c.execute(request)\n",
" res = c.fetchall()\n",
" for i in range(len(res)):\n",
" res[i] = datetime.datetime.strptime(list(res[i])[0][2:10],\"%y-%m-%d\").date()\n",
" return res\n",
"\n",
"def insertCurrencyMarketElement(currency : str, element : dict):\n",
" \"\"\"Insert a new currency market element in the corresponding table\n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency\n",
" element (dict): new element to add in the table , this dict must contaign definition for {Date,Open,Hight,Low,Close,Volume}\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" tableName = cleanNameForSQL(currency)\n",
" request = f'''INSERT INTO {tableName}_MARKET (Date,Open,High,Low,Close,Volume) VALUES \n",
" ('{element['Date']}',{element['Open']},{element['High']},{element['Low']},{element['Close']},{element['Volume']})'''\n",
" c .execute(request)\n",
" db.commit()\n",
"\n",
"def printCurrencyMarketElement(currency : str , currencyId : int):\n",
" \"\"\"Print an element of the currency table by its id \n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency \n",
" currencyId (int): id of the currency in the table \n",
" \"\"\"\n",
" data = getCurrencyMarketElementById(currency , currencyId)\n",
" header = f\"-------- Value for {currency} ID°{currencyId} --------\\n\"\n",
" nHeader = len(header)\n",
" print(header)\n",
" print(f\"Date : {data[1]}\")\n",
" print(f\"Open : {data[2]}\")\n",
" print(f\"High : {data[3]}\")\n",
" print(f\"Low : {data[4]}\")\n",
" print(f\"Close : {data[5]}\")\n",
" print(f\"Volume : {data[6]}\")\n",
" footer = \"\\n\"\n",
" for i in range(nHeader): footer += \"-\"\n",
" print(footer)\n",
"\n",
"#No Modify or Delete function debause they currently don't seem usefull "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h4>Forecast Table</h4>\n",
"There is one forcast table for each currency fetched. It contaigns :\n",
"<ul><li><b>Date</b> ~> date of the forecast</li>\n",
"<li><b>Open</b> ~> forecast value of the stock when the market open</li>\n",
"<li><b>High</b> ~> forecast highest value reach by the stock during the market time</li> \n",
"<li><b>Low</b> ~> forecast lowest value reach by the stock during the market time</li>\n",
"<li><b>Close</b> ~> forecast value of the stock when the market close </li>\n",
"</ul>\n",
"It is possible to :\n",
"<ul><li>Create / Fill <b>table</b></li>\n",
"<li>Insert / Get / Modify / Delete / Print <b>table elements</b></li></ul>"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"def createCurrencyForecastTable(currencyName : str):\n",
" \"\"\"Create a new table if it dont alredy exist to contaign market forecast prevision of a currency\n",
"\n",
" Args:\n",
" currencyName (str): name of the currency for which a new table is created \n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" tableName = cleanNameForSQL(currencyName)\n",
" #each lign in the table represent a new day and contaign :\n",
" # Date : date ~> forecast date of the information present\n",
" # Open : float ~> forecast value of the stock when the market open\n",
" # High : float ~> forecast highest value reach by the stock during the market time \n",
" # Low : float ~> forecast lowest value reach by the stock during the market time\n",
" # Close : float ~> forecast value of the stock when the market close \n",
" createNewCurrencyForecastTable = f'''CREATE TABLE IF NOT EXISTS {tableName}_FORECAST \n",
" ([Id] INTEGER PRIMARY KEY AUTOINCREMENT,\n",
" [Date] TEXT,\n",
" [Open] NUMBER,\n",
" [High] NUMBER,\n",
" [Low] NUMBER,\n",
" [Close] NUMBER)'''\n",
" c.execute(createNewCurrencyForecastTable)\n",
" db.commit()\n",
"\n",
"def getCurrencyForecastElementById(currency : str , elementId : int ) -> dict :\n",
" \"\"\"Get an from a currency forecast table by its id \n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency , used to get the information with the API\n",
" elementId (int): id of the element to get \n",
"\n",
" Returns:\n",
" dict: element of the table which id is elementId \n",
" \"\"\"\n",
" tableName = cleanNameForSQL(currency)\n",
" request = f'''SELECT * FROM {tableName}_FORECAST WHERE Id = {elementId} '''\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" c.execute(request)\n",
" return c.fetchone()[0]\n",
"\n",
"def insertCurrencyForecastElement(currency : str, element : dict):\n",
" \"\"\"Insert a new currency forecast element in the corresponding table \n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency\n",
" element (dict): new element to add in the table , this dict must contaign definition for {Date,Open,Hight,Low,Close,Volume}\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" tableName = cleanNameForSQL(currency)\n",
" request = f'''INSERT INTO {tableName}_FORECAST (Date,Open,High,Low,Close,Volume) VALUES \n",
" ('{element['Date']}',{element['Open']},{element['High']},{element['Low']},{element['Close']})'''\n",
" c .execute(request)\n",
" db.commit()\n",
"\n",
"def modifyCurrencyForecastElement(currency : str , elementId : int, newElement : dict):\n",
" \"\"\"Modify an element in the currency forecast table, the element to modify must be enteriely rewrite\n",
"\n",
" Args:\n",
" currency (str): id of the currency to modify in the currency table\n",
" elementId (int): id of the currency to modify in the currency table\n",
" newElement (dict): dictionnary contaigning all the information of the currency even the ones not changed\n",
" \"\"\"\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" #All the information MUST be given in the newMetaData dict even the id and the ones not changed\n",
" tableName = cleanNameForSQL(currency)\n",
" request = f'''UPDATE {tableName}_FORECAST \n",
" SET \n",
" Date={newElement['Id']},\n",
" Open='{newElement['Open']}', \n",
" High='{newElement['High']}', \n",
" Low={newElement['Low']},\n",
" Close={newElement['Close']},\n",
" WHERE Id = {elementId}'''\n",
" c.execute(request)\n",
" db.commit()\n",
"\n",
"def deleteCurrencyForecastElement(currency : str , currencyId : int):\n",
" \"\"\"Delete an element from the currency forecast table\n",
"\n",
" Args:\n",
" currency (str): yfinance currency name \n",
" currencyId (int): id of the currency to delete from the currency table\n",
" \"\"\"\n",
" tableName = cleanNameForSQL(currency)\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" request = f'''DELETE FROM {tableName}_FORECAST \n",
" WHERE currencyId = {currencyId}'''\n",
" c.execute(request)\n",
" db.commit()\n",
"\n",
"def printCurrencyForecastElement(currency : str , currencyId : int):\n",
" \"\"\"Print an element of the currency forecast table by its id \n",
"\n",
" Args:\n",
" currency (str): yahoo finance name of the currency \n",
" currencyId (int): id of the currency in the table \n",
" \"\"\"\n",
" data = getCurrencyForecastElementById(currency , currencyId)\n",
" header = f\"-------- Forecast for {currency} ID°{currencyId} --------\\n\"\n",
" nHeader = len(header)\n",
" print(header)\n",
" print(f\"Date : {data[1]}\")\n",
" print(f\"Open : {data[2]}\")\n",
" print(f\"High : {data[3]}\")\n",
" print(f\"Low : {data[4]}\")\n",
" print(f\"Close : {data[5]}\")\n",
" footer = \"\\n\"\n",
" for i in range(nHeader): footer += \"-\"\n",
" print(footer)\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h4>Initialization DB Function</h4>\n",
"<p>Scipt to initialize the database</p>"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"def setDb(currencies : list):\n",
" \"\"\"Initialize the database\n",
"\n",
" Args:\n",
" currencies (list): list of currencies to add during the db initailization\n",
" \"\"\"\n",
" createCurrencyMetadataTable()\n",
" for currency in currencies:\n",
" print(currency)\n",
" insertCurrencyMetadataElement(currency)\n",
" createCurrencyMarketTable(currency)\n",
" fillCurrencyMarketTable(currency,period = \"1y\")\n",
" createCurrencyForecastTable(currency)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"BTC-USD\n",
"INSERT INTO CURRENCIES_METADATA (currencyName,instrumentType,regularMarketTime,gtmOffset,timezone)\n",
" VALUES (\n",
" 'BTC-USD',\n",
" 'CRYPTOCURRENCY',\n",
" '2023-04-25 11:45:00+00:00',\n",
" 0,\n",
" 'UTC')\n",
"ETH-USD\n",
"INSERT INTO CURRENCIES_METADATA (currencyName,instrumentType,regularMarketTime,gtmOffset,timezone)\n",
" VALUES (\n",
" 'ETH-USD',\n",
" 'CRYPTOCURRENCY',\n",
" '2023-04-25 11:45:00+00:00',\n",
" 0,\n",
" 'UTC')\n",
"TTE.PA\n",
"INSERT INTO CURRENCIES_METADATA (currencyName,instrumentType,regularMarketTime,gtmOffset,timezone)\n",
" VALUES (\n",
" 'TTE.PA',\n",
" 'EQUITY',\n",
" '2023-04-25 13:32:36+02:00',\n",
" 7200,\n",
" 'CEST')\n"
]
}
],
"source": [
"#create db\n",
"try:\n",
" os.remove(\"mydatabase.db\") #delete database if it exixts \n",
"except:\n",
" pass # if it not exists just go next\n",
"setDb(currencies)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h4>Display stock chart</h4>"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"tags": []
},
"outputs": [
{
"ename": "OperationalError",
"evalue": "no such table: BTCUSD_MARKET",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mOperationalError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[14], line 14\u001b[0m\n\u001b[1;32m 11\u001b[0m plt\u001b[39m.\u001b[39mplot(dateData,closeData)\n\u001b[1;32m 13\u001b[0m \u001b[39m# displayStockChart(\"BTC-USD\",\"5y\")\u001b[39;00m\n\u001b[0;32m---> 14\u001b[0m displaySimplerStockChart(\u001b[39m\"\u001b[39;49m\u001b[39mBTC-USD\u001b[39;49m\u001b[39m\"\u001b[39;49m)\n",
"Cell \u001b[0;32mIn[14], line 5\u001b[0m, in \u001b[0;36mdisplaySimplerStockChart\u001b[0;34m(currency)\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mdisplaySimplerStockChart\u001b[39m(currency: \u001b[39mstr\u001b[39m):\n\u001b[1;32m 4\u001b[0m plt\u001b[39m.\u001b[39mstyle\u001b[39m.\u001b[39muse(\u001b[39m\"\u001b[39m\u001b[39mbmh\u001b[39m\u001b[39m\"\u001b[39m)\n\u001b[0;32m----> 5\u001b[0m closeData \u001b[39m=\u001b[39m getCloseCurrencyMarketElement(currency)\n\u001b[1;32m 6\u001b[0m dateData \u001b[39m=\u001b[39m getDateCurrencyMarketElement(currency)\n\u001b[1;32m 7\u001b[0m plt\u001b[39m.\u001b[39mfigure(figsize\u001b[39m=\u001b[39m(\u001b[39m16\u001b[39m,\u001b[39m4\u001b[39m))\n",
"Cell \u001b[0;32mIn[6], line 112\u001b[0m, in \u001b[0;36mgetCloseCurrencyMarketElement\u001b[0;34m(currency)\u001b[0m\n\u001b[1;32m 110\u001b[0m db \u001b[39m=\u001b[39m sql\u001b[39m.\u001b[39mconnect(\u001b[39m\"\u001b[39m\u001b[39mmydatabase.db\u001b[39m\u001b[39m\"\u001b[39m)\n\u001b[1;32m 111\u001b[0m c \u001b[39m=\u001b[39m db\u001b[39m.\u001b[39mcursor()\n\u001b[0;32m--> 112\u001b[0m c\u001b[39m.\u001b[39;49mexecute(request)\n\u001b[1;32m 113\u001b[0m res \u001b[39m=\u001b[39m c\u001b[39m.\u001b[39mfetchall()\n\u001b[1;32m 114\u001b[0m \u001b[39mfor\u001b[39;00m i \u001b[39min\u001b[39;00m \u001b[39mrange\u001b[39m(\u001b[39mlen\u001b[39m(res)):\n",
"\u001b[0;31mOperationalError\u001b[0m: no such table: BTCUSD_MARKET"
]
}
],
"source": [
"import matplotlib.pyplot as plt \n",
"\n",
"def displaySimplerStockChart(currency: str):\n",
" plt.style.use(\"bmh\")\n",
" closeData = getCloseCurrencyMarketElement(currency)\n",
" dateData = getDateCurrencyMarketElement(currency)\n",
" plt.figure(figsize=(16,4))\n",
" plt.title(f\"{currency} Closing Prices\")\n",
" plt.xlabel('Date')\n",
" plt.ylabel('Closing Price')\n",
" plt.plot(dateData,closeData)\n",
"\n",
"# displayStockChart(\"BTC-USD\",\"5y\")\n",
"displaySimplerStockChart(\"BTC-USD\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"<h3>Database Test</h3>"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"def GeneralDatabaseTest():\n",
" #Test if db exists\n",
" print(\"--- Database General Test ---\")\n",
" if os.path.exists(\"./mydatabase.db\") : \n",
" print(\"Test 1 passed : Database exist\")\n",
" else : \n",
" print(\"Test failled : No database named `mydatabase.db` created\")\n",
" return 0\n",
" return 1\n",
"\n",
"def CurrenciesMetadatGeneralTest():\n",
" \"\"\"Assert the structure of the CURRENCIES_METADATA is good\"\"\"\n",
" print(\"\\n--- CURRENCIES_METADATA General Test ---\")\n",
" #Test if CURRENCIES_METADATA structure is good\n",
" db = sql.connect(\"mydatabase.db\")\n",
" c = db.cursor()\n",
" request ='''SELECT * FROM CURRENCIES_METADATA'''\n",
" c.execute(request)\n",
" elem = c.fetchone()\n",
" #First test number of element\n",
" if len(elem) == 6 : \n",
" print(\"Test 2 passed : Number of element in table good\")\n",
" else : \n",
" print(\"Test failed : The number of element in the CURRENCIES_METADATA table isn't correct\")\n",
" return 0\n",
" #Then test element types\n",
" types = (int , str , str , int , int , str)\n",
" testResult = True\n",
" for i in range(len(elem)):\n",
" if type(elem[i]) != types[i]:\n",
" testResult = False\n",
" print(f'Test failed : the type of the {i} element doesn\\'t correspond, it\\'s a {type(elem[i])} but should be a {types[i]}')\n",
" return 0\n",
" if testResult: print(\"Test 3 passed : Element types good\")\n",
" return 1\n",
"\n",
"def CurrenciesMetadatGetTest():\n",
" \"\"\"Assert the first element of the CURRENCIES_METADATA table correspond to the `BTC-USD` one\"\"\"\n",
" print(\"\\n--- CURRENCIES_METADATA GET Method Test ---\")\n",
" btcusd = (1 , \"BTC-USD\" , \"CRYPTOCURRENCY\" , -1 , 0 , \"UTC\")\n",
" resget = getCurrencyMetadataElement(1)\n",
" t = np.column_stack((resget , btcusd))\n",
" for i , j in t:\n",
" if not i == j and j == -1:\n",
" print(f'Test failed : the element get by the method does not correspond to the BTC-USD one the values {i} should be {j}')\n",
" return 0\n",
" print(\"Test 4 passed : The Element Fetched Correspond\")\n",
" return 1\n",
"\n",
"def CurrenciesMetadatInsertTest():\n",
" \"\"\"Assert the neawly inserted element of the CURRENCIES_METADATA table is inserted and correct\"\"\"\n",
" print(\"\\n--- CURRENCIES_METADATA INSERT Method Test ---\")\n",
" nElem = len(currencies)\n",
" newElem = (nElem , \"newElem\" , \"CRYPTOCURRENCY\" , -1 , 0 , \"UTC\")\n",
" insertCurrencyMetadataElement(\"TPA\")\n",
" resget = getCurrencyMetadataElement(nElem)\n",
" t = np.column_stack((resget , newElem))\n",
" for i , j in t:\n",
" if not i == j and j == -1:\n",
" print(f'Test failed : the element inserted isn\\'t correct : the values {i} should be {j}')\n",
" return 0\n",
" print(\"Test 5 passed : The Element Is Inserted and is Correct\")\n",
" return 1\n",
"\n",
"def TestCurrencyTable():\n",
" \"\"\"Assert all the CURRENCIES_METADATA table structure and method are functionnal\"\"\"\n",
" return CurrenciesMetadatGeneralTest() and CurrenciesMetadatGetTest() and CurrenciesMetadatInsertTest()\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--- Database General Test ---\n",
"Test 1 passed : Database exist\n",
"\n",
"--- CURRENCIES_METADATA General Test ---\n"
]
},
{
"ename": "TypeError",
"evalue": "object of type 'NoneType' has no len()",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[12], line 10\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39mTests have Failed\u001b[39m\u001b[39m\"\u001b[39m)\n\u001b[1;32m 8\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39m0\u001b[39m\n\u001b[0;32m---> 10\u001b[0m TestDatabase()\n",
"Cell \u001b[0;32mIn[12], line 6\u001b[0m, in \u001b[0;36mTestDatabase\u001b[0;34m()\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39mTests have Failed\u001b[39m\u001b[39m\"\u001b[39m)\n\u001b[1;32m 5\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39m0\u001b[39m\n\u001b[0;32m----> 6\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m TestCurrencyTable(): \n\u001b[1;32m 7\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39mTests have Failed\u001b[39m\u001b[39m\"\u001b[39m)\n\u001b[1;32m 8\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39m0\u001b[39m\n",
"Cell \u001b[0;32mIn[11], line 67\u001b[0m, in \u001b[0;36mTestCurrencyTable\u001b[0;34m()\u001b[0m\n\u001b[1;32m 65\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mTestCurrencyTable\u001b[39m():\n\u001b[1;32m 66\u001b[0m \u001b[39m \u001b[39m\u001b[39m\"\"\"Assert all the CURRENCIES_METADATA table structure and method are functionnal\"\"\"\u001b[39;00m\n\u001b[0;32m---> 67\u001b[0m \u001b[39mreturn\u001b[39;00m CurrenciesMetadatGeneralTest() \u001b[39mand\u001b[39;00m CurrenciesMetadatGetTest() \u001b[39mand\u001b[39;00m CurrenciesMetadatInsertTest()\n",
"Cell \u001b[0;32mIn[11], line 21\u001b[0m, in \u001b[0;36mCurrenciesMetadatGeneralTest\u001b[0;34m()\u001b[0m\n\u001b[1;32m 19\u001b[0m elem \u001b[39m=\u001b[39m c\u001b[39m.\u001b[39mfetchone()\n\u001b[1;32m 20\u001b[0m \u001b[39m#First test number of element\u001b[39;00m\n\u001b[0;32m---> 21\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mlen\u001b[39;49m(elem) \u001b[39m==\u001b[39m \u001b[39m6\u001b[39m : \n\u001b[1;32m 22\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39mTest 2 passed : Number of element in table good\u001b[39m\u001b[39m\"\u001b[39m)\n\u001b[1;32m 23\u001b[0m \u001b[39melse\u001b[39;00m : \n",
"\u001b[0;31mTypeError\u001b[0m: object of type 'NoneType' has no len()"
]
}
],
"source": [
"def TestDatabase():\n",
" \"\"\"Assert all the database is functionnal\"\"\"\n",
" if not GeneralDatabaseTest(): \n",
" print(\"Tests have Failed\")\n",
" return 0\n",
" if not TestCurrencyTable(): \n",
" print(\"Tests have Failed\")\n",
" return 0\n",
"\n",
"TestDatabase()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.2"
},
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}

3
Db-Script/database.py Normal file
View file

@ -0,0 +1,3 @@
##This mist include all fetch function and also init and insert function for nn training annd testing result
import numpy

BIN
Db-Script/db Executable file

Binary file not shown.

BIN
Db-Script/robotgowest.db Normal file

Binary file not shown.

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

BIN
Image_Diagram/chaien.PNG Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

View file

@ -71,7 +71,8 @@ f = np.fft.fftfreq(Fe,tstep) #axe des abscisses: fréquence
#Définition des courbes des plots #Définition des courbes des plots
ax1.plot(t,y) # print y(t) ax1.plot(t,y) # print y(t)
ax2.plot(f,abs(sp)) ax2.plot(f,abs(sp))#ax1 = plt.axes()
#Lancement de l'affichage du plot #Lancement de l'affichage du plot
plt.show() plt.show()

111
Py-Script/courbe.py Normal file
View file

@ -0,0 +1,111 @@
from pickletools import long1
from secrets import choice
from turtle import goto
import numpy as np
import matplotlib.pyplot as plt
import csv
#"C:\Users\quent\OneDrive\Bureau\ENSC\TransD\Framboisier\02400001.TXT"
#! /usr/bin/env python3
# -*- coding: UTF-8 -*-
# x = tab[:,0]
# y = tab[:,1]
# print(tab)
# return x , y
#fonction choix du capteur à afficher :
#bool true si capteur veux être utiliser sinon false (tableau)
def function():
while True:
print("\n")
choice = input("(1) Choisis un fichier csv de 1 à 1862: ")
print("\n")
if float(choice) in range(1, 1863):
rawdata = '../RawDataFiles/RawData' + choice + '.csv' #Définit le fichier à ouvrir
choice2 = input("(2) Choisir un capteur à afficher en particulier dans le tableau? oui/non \n")
print("\n")
if choice2 == "oui":
while True:
ccapteur = int(input("(3) Choisis parmis les capteurs (1, 2, 3, 4, 5, 6, 7, 8): "))
print("\n")
if ccapteur in (1,2,3,4,5,6,7,8): #Affiche le capteur sélectionner
with open(rawdata, newline='') as csvfile:
reader = csv.reader(csvfile, quoting=csv.QUOTE_NONNUMERIC)
x = [row for row in reader]
int_x = np.array(x, int)
column_index = [ccapteur + 1]
for col in column_index:
column_values = np.empty(0)
for array in range(len(int_x)):
column_values = np.append(column_values, np.array(int_x[array][col-1]))
plt.plot(column_values)
plt.show()
exit()
elif ccapteur != (1,2,3,4,5,6,7,8):
print("{} n'est pas un nombre valide.\n".format(int(ccapteur)))
elif choice2 == "non": #Ouvre le csv file et affiche tout les capteurs
with open(rawdata, newline='') as csvfile:
reader = csv.reader(csvfile, quoting=csv.QUOTE_NONNUMERIC)
x = [row for row in reader]
int_x = np.array(x, int)
column_index = [2,3,4,5,6,7,8,9]
for col in column_index:
column_values = np.empty(0)
for array in range(len(int_x)):
column_values = np.append(column_values, np.array(int_x[array][col-1]))
plt.plot(column_values)
plt.show()
break
elif choice > '1862' or choice < '1':
print("{} n'est pas un nombre valide.\n".format(choice))
#Fonction d'affichage du graphique
def printFig(axe_x , axe_y , title = "new plot" , xlabel = "Temps" , ylabel = "Tension"):
plt.plot(axe_x , axe_y)
plt.xlabel(xlabel)
plt.ylabel(ylabel)
plt.title(title)
plt.show()
return 0
#print(x)
#print("y = " + str(y))
def final():
function()
#printFig()
final()
def function():
for i in range(1, 10):
rawdata = 'Code-C/RawDataFiles/RawData' + str(i) + '.csv' #Définit le fichier à ouvrir
with open(rawdata, newline='') as csvfile:
reader = csv.reader(csvfile, quoting=csv.QUOTE_NONNUMERIC)
x = [row for row in reader]
int_x = np.array(x, int)
for j in range(1,len(int_x[0])):
aver = average(int_x[:,j])
print("I = " , i ," J = " ,j ," aver = " , aver , '\0')
def average(x):
res = 0
for i in range(len(x)):
res += x[i]
return res / len(x)
function()

28
README.md Normal file
View file

@ -0,0 +1,28 @@
# [Robot Go West](https://projets.cohabit.fr/redmine/projects/communication-racinaire/wiki/Robot_Go-West)
### Introduction
This projet is a part of larger projet [_Communication Racinaire_](https://projets.cohabit.fr/redmine/projects/communication-racinaire/wiki/Wiki#section-8) which aims to drive plante into the direction of the sun.
When a plant received the solar radiation, a **difference of potential** will be created inside the plant. This physical reaction interest us to **measure** it and **interpret** the signal to drive the robot.
![Diagram](https://git.cohabit.fr/pgp/Traitement-signal-plantes/src/branch/Data/Image_Diagram/chaien.PNG)
On the diagram we can see a "**Pi**" symbole which represents the micro-controller who calculate and drive the robot, who will make the **C sources** work. After many hours of working or test phase the microcontroller create and store data interpretable by the **Python script**.
## Embedded C sources
![Graphical representation](https://git.cohabit.fr/pgp/Traitement-signal-plantes/src/branch/Data/Image_Diagram/ResumeWorkingDiagram.PNG)
The scheme represents well the program operation on the raspberry, the diagram is read from left to right.
**The first** represent the stream from vegetalsignal's captor through the USB port. This part block us because we don't have stream for now with the captor then we have create a thread to similate it. The stream send Raw data correspond to the electrique tension generated by the plant.
**The second** part with the thread storeRawData catch the raw data and store it in the memory in several files.(Usable data for python script)
**The third** part is used to gather data files and calculate to have pertinante data (Power or GrowthRate) and store it in file.
**The fourth** is not yet done, the thread drive will recovers the power and/or the gorwthRate data recognize events in the data and drive the robot.
## Graphic visualization Python script
The Python script is used to print the tension data on a graph.